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Any  computerized  picture  processing  system  can  gener- 
ally be  divided  into  four  major  units:   a  picture  digitizer, 
an  object  extractor,  a  feature  extractor  and  a  classifier. 
This  dissertation  is  concerned  mainly  with  new  approaches  to 
object  extraction  and  feature  extraction. 

Three  information  handling  methods  have  been  developed 
which  may  be  used  to  mechanize  the  extraction  of  objects 
from  multi-level  pictures.   These  methods  are  those  of  the 
gradient  analysis,  the  contour  analysis  and  the  graph  theory 
approach.   In  the  gradient  analysis  method,  a  locally  optimal 
threshold  is  used  to  find  the  boundary  points.   In  this  new 
approach,  high  efficiency  is  achieved  because  the  tentative 
boundary  paths  are  simultaneously  found.   Then,  after  filling 
the  gaps  along  the  boundaries  or  at  the  intersections  and 
removing  the  tails,  boundary  segments  are  found.   Rules  are 
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set  to  combine  the  boundary  segments  in  order  to  decompose 
the  overlapping,  self-folding  and  touching  objects  in  an 
area  picture.   The  contour  analysis  method  is  developed  on 
the  assumption  that  the  threshold  used  for  transfering  a 
multi-level  picture  to  a  binary  picture  is  approximately 
constant  in  a  window.   This  method  permits  very  successful 
object  extraction  for  a  multi-level  picture  with  selective 
deletion  of  nonboundary  contours.   In  the  graph  theory  ap- 
proach, a  multi-level  picture  is  transfered  to  a  weighted 
graph.   An  MSF  (Minimal  Spanning  Forest)  of  the  weighted  graph 
is  then  found.   By  finding  the  principal  paths  of  a  tree  in 
the  MSF,  the  skeleton  of  the  object  corresponding  to  this  tree 
can  then  be  found. 

The  boundary  contour  of  an  object  has  been  encoded  by 
a  sequence  of  octal  chain  codes.   A  local  feature  extractor 
has  been  designed  to  find  the  area,  centroid,  shape,  princi- 
pal axis  direction  and  the  elongation  index  of  an  object  with 
the  knowledge  of  the  sequence  of  octal  chain  codes  of  the 
boundary  contour.   A  global  feature  extractor  has  been  de- 
signed to  find  the  inclusion  relationship  among  objects  and 
the  distribution  of  objects  in  a  picture.   The  inclusion 
relationship  is  represented  by  a  Kasse  graph.   The  distribu- 
tion of  objects  may  be  represented  by  an  MST  (Minimal  Span- 
ning Tree)  . 

The  newly  designed  object  extractor  and  the  feature 
extractor  methods  have  been  tested  by  analysis  of  the 
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information  in  the  pictures  of  chromosomes,  skin  cells  and 
blood  cells.   In  evaluation  of  chromosome  pictures,  the 
major  tasks  are  to  identify  and  catagorize  all  chromosomes. 
In  analysis  of  the  histological  skin  cell  photomicrographs, 
the  problem  is  to  find  the  structure  of  cells  in  epidermis 
in  order  to  detect  the  degree  of  the  malignancy  of  possible 
tumors.   In  evaluation  of  the  blood  cell  photomicrographs, 
the  goal  is  to  obtain  the  histogram  of  the  blood  cell  photo- 
intensities  in  order  to  reveal  critical  diagnostic  informa- 
tion.  In  each  of  these  three  evaluation  tests  very  promis- 
ing results  were  achieved  by  the  use  of  combinations  of  the 
new  techniques.   A  more  complete  computerized  picture  proces 
sing  system  is  suggested,  as  an  extension  of  the  newly 
developed  techniques. 
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CHAPTER  I 
INTRODUCTION 

Picture  processing  is  a  process  which  transfers  scenes 
to  descriptions.  For  example,  when  a  picture  processor 
"sees"  a  writing  "Q",  it  should  be  able  to  tell  that  it  is 
the  Chinese  character  for  "sun."  When  a  picture  processor 
"sees"  the  scene  as  shown  in  Figure  1.1,  it  should  tell 
that  there  are  two  cubes,  A  and  B,  in  the  scene,  where  cube 
A  is  in  front  of  cube  B. 


Figure  1.1.   A  scene  of  two  cubes 


The  processor  performs  two  main  functions:   the  first 
is  "to  see"  and  the  second  is  "to  give  the  descriptions" 


from  what  is  "seen."   "To  see"  is  the  process  usually  called 
object  extraction  from  the  scene.   "To  give  the  descriptions" 
from  what  has  been  "seen"  includes  feature  extraction  and 
identification.   In  general,  human  beings  are  the  best  pic- 
ture processors  up  to  the  present.   One  drawback  of  man's 
ability  as  a  picture  processor  is  that  his  visual  system  is 
easily  tired.   Mechanization  of  the  picture  processing  be- 
came possible  after  the  invention  of  the  modern  computer. 
This  mechanization  is  very  desirable  as  it  frees  manpower 
from  routine  visual  tasks. 

There  are  two  principal  types  of  pictures  encountered 
in  everyday  life.   One  is  the  picture  of  three-dimensional 
objects.   This  type  of  picture  is  the  projection  of  the 
three-dimensional  object  on  a  picture  plane.   The  projection 
is  supposed  to  exhibit  the  depth  information.   Several  re- 
searchers^   J  have  conducted  research  dealing  with  this 
type  of  three-dimensional  picture.   The  other  type  of  pic- 
ture is  two-dimensional.   Two-dimensional  pictures  are 
either  artificial  pictures,  such  as  characters^  '  '    and 
maps,  '    or  natural  images  whose  depth  information  is  not 
important  and  almost  cannot  be  seen  in  the  picture  planes, 
such  as  pictures  of  particle  tracks  in  the  bubble  cham- 
ber/ '  '   fingerprints  *•  ^    and  cell  images.*-   *   ^   From 
here  on,  "picture  processing"  means  the  mechanization  of 
picture  processor  unless  otherwise  specified. 


A  picture  processing  system  can  generally  be  divided 
into  four  parts:   a  picture  digitizer,  an  object  extractor, 
a  feature  extractor  and  a  classifier.   The  picture  digitizer 
and  the  object  extractor  perform  the  function  of  "seeing." 
The  feature  extractor  and  the  classifier  perform  the  func- 
tion of  "giving  the  descriptions."   Figure  1.2  is  a  block 
diagram  of  a  general  picture  processing  system. 


optical 
picture 


picture 
digitizer 

object 
extractor 

feature 
extractor 

classi- 
fier 

Figure  1.2.   A  block  diagram  of  a  general  picture  proces 
sing  system. 


1.1.   A  Survey  of  Research  in  the  Area 
of  Picture  Processing 

A  brief  survey  of  the  area  of  picture  processing  is 
presented  in  this  section. 


1.1.1.   Picture  Digitizer 

A  picture  digitizer  transforms  the  data  of  an  image 
to  a  digitized  form  which  is  accessible  by  a  digital  com- 
puter.  An  optical  picture  can  be  represented  mathematically 
as  a  real  function  f  on  a  picture  plane  D,  which  is  a  simply 
connected  subset  of  the  real  plane,  f:  D->R,  where  R  is  the 
set  of  the  intensity  values  of  the  picture  points.   There 


•   i       *       *■   (12)     .  . 
are  two  principal  ways  to  quantize v  *    a  picture  plane: 

the  hexagonal  grid  and  the  rectangular  grid.   Figure  1.3 
shows  the  two  types  of  grids.   Hexagonal  grids  have  the 
advantage  of  having  six  neighboring  picture  points,  which 
are  nearest  to  p,  for  every  picture  point  p.   They  have  the 
drawback  of  being  based  on  an  uncommon,  non-orthogonal  co- 
ordinate system.   The  rectangular  grids  contain  only  four 
neighboring  picture  points,  which  are  nearest  to  p,  for 
every  picture  point  p,  but  it  is  very  easy  to  access  every 
picture  point. 


(a)  (b) 


Figure  1.3.   Examples  of  (a)  hexagonal  grid  and  (b)  rec- 
tangular grid. 


The  rectangular  grid  forms  an  orthogonal  coordinate 
system.   Hence  the  picture  plane  becomes  I   =1X1, 
where  I   and  I   are  subsets  of  the  integer  set.   From  here 
on,  all  digitized  pictures  use  rectangular  grids.   The  in- 
tensities of  a  quantized  picture  are  quantized  into  n 
levels.   Usually  n  is  set  equal  to  2   because  this  maximizes 


storage  efficiency  within  the  bit-oriented  digital  computers. 
A  2  -level  picture  is  called  a  k-bit  picture.   If  k  =  1, 
the  digitized  picture  is  called  a  binary  picture.  Ann-level 
digitized  picture  is  a  mapping  g:  I->-N,  where  N  =  (0,1,..., 
n-1)  is  the  set  of  quantized  intensity  values.   A  digitized 
picture  can  also  be  represented  as  a  matrix.   The  location 
of  a  picture  point  is  specified  by  the  location  of  the  ele- 
ment in  the  matrix.   The  intensity  of  a  picture  point  is 
indicated  by  the  value  of  the  corresponding  element  in  the 
matrix.   Figure  1.4  is  a  binary  picture  of  a  numeral  "6" 
represented  as  this  matrix  form. 

A  picture  digitizer  performs  a  transformation  from  an 
f  mapping  to  a  new  mapping  g.   A  complete  picture  digitizing 
system,  PIDAC  (Pictorial  Data  Acquisition  Computer) ,  has 
been  implemented  at  the  CIR  (Center  for  Informatics  Research) 
in  the  University  of  Florida.   The  PIDAC,  which  is  a  modifi- 
cation  of  the  FIDAC^  J    (Film  Input  to  Digital  Automatic 
Computer)  system,  is  one  of  the  better  picture  digitizers 
available  today.   It  consists  of  a  CRT,  two  lenses,  a  photo- 
multiplier,  an  a-d  converter  and  a  scan  control  unit.   The 
digitized  pictures  are  stored  on  a  magnetic  tape.   The 
PIDAC  can  alternately  be  interfaced  with  a  digital  computer 
to  store  the  digitized  form  from  a  picture.   The  maximum 
spatial  resolution  of  the  PIDAC  is  1,240  spots  along  the 
long  axis  and  800  lines  per  35  mm  film.   The  maximum  digi- 
tized level  of  the  PIDAC  is  24  =  64.   Very  good  8-level 


000000000000000000000000 
000000001110000000000000 
000000011110000000000000 
000000011110000000000000 
000000111110000000000000 
000000111100000000000000 
000001111100000000000000 
000001111000000000000000 
000011111000000000000000 
000011111000000000000000 
000011110000000000000000 
000111110000000000000000 
000111111111100000000000 
000111111111111000000000 
000111111111111000000000 
000011111111111100000000 
000011111000111100000000 
000011111000111100000000 
000001111111111100000000 
000000111111111100000000 
000000011111111000000000 
000000000111100000000000 
000000000  000000000000000 
000000000000000000000000 


Figure  1.4.   A  binary  picture  of  numeral  M6 . " 


pictures  can  be  achieved  by  the  PIDAC.   The  scanning  speed 
of  the  PIDAC  is  .3  sec/picture.   Figure  1.5  is  a  portion  of 
an  8-level  skin  cell  picture  obtained  from  PIDAC. 

Once  a  digitized  picture  is  obtained,  the  picture  data 
are  then  accessible  by  digital  computers.   The  next  process 
is  the  extraction  of  objects  from  the  picture. 

1.1.2.   Object  Extraction 

There  are  mainly  three  methods  used  in  extracting  ob- 
jects from  the  scene.   The  first  method  finds  the  boundaries 
of  objects  and  then  decomposes  objects  from  boundaries. 
The  second  method  finds  the  thresholds  to  transfer  a  multi- 
level picture  to  a  binary  picture  and  then  finds  the  con- 
tours of  the  binary  pictures.   The  third  method  finds  the 
clusters  in  a  picture  and  considers  each  cluster  as  an  ob- 
ject. 

There  are  two  main  approaches  used  to  find  the  bounda- 
ries of  objects.   One  approach  finds  the  enhanced  picture 
first  and  then  finds  the  boundaries.   The  enhanced  pictures 
can  be  found  either  from  the  spatial  domain  or  from  the 
spatial  frequency  domain.   To  find  the  enhanced  pictures 
directly  from  the  picture  plane  (spatial  domain) ,  the  most 
frequently  used  methods  are  the  gradient  method^  '     and 
the   Laplacian  method.  *■  '      By  the  gradient  method,  each 
picture  point  in  an  enhanced  picture  is  set  to  have  a  value 
equal  to  the  gradient  of  intensity  at  that  picture  point. 
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Figure  1.5.   A  digitized  picture  of  a  portion  of  a  skin 
cell  picture. 


Since  the  data  are  digitized,  approximation  of  gradient  is 

f  11  2 

used,  such  as    h(i,j)  =  [  (g(i , j ) -g(i  +  l ,  j +1) )   + 

2  1/2 
(g(i+l,jl-  g(i,j+l))  ]    ,  which  is  a  very  good  approxima- 
tion except  that  square  root  calculation  is  involved. 
Approximation  of  the   Laplacian  function  is  needed  for 
digitized  data,  such  as ^   J  L(i,j)  =  g(i-l , j ) +g (i+1 , j ) + 
g(i  ,  j -1) +g(i , j  +  1) -4g  (i  ,  j ) .   The  function  L  then  represents 
the  enhanced  picture.   The  enhanced  picture  will  normally 
have  high  values  at  the  boundaries.   To  find  the  enhanced 
picture  from  the  spatial  frequency  domain,  *-  *    the  picture 
f  is  transformed  to  a  Fourier  spectrum  F  first.   A  high- 
pass  filter  H  is  applied  to  enhance   values  of  F  at  high 
frequencies  relative  to  those  at  low  frequencies.   The  in- 
verse Fourier  transformation  of  FH  is  the  corresponding  en- 
hanced picture.   After  the  enhanced  picture  is  found,  a 
threshold  is  then  set  to  find  the  boundary  points.   Boundary 
points  are  connected  by  a  multi-step  process.  '      The  other 
approach  finds  the  boundaries  by  use  of  a  matched  filter 
which  can  extract  the  boundaries  directly  from  the  picture 
g.   The  purpose  of  the  edge  operator  used  by  Huechel^   '  is 
to  fit  an  ideal  edge  element  to  any  empirically  obtained 
edge  element.   In  scanning  the  picture,  when  an  edge  is 
found  by  the  edge  operator,  scanning  is  interrupted  and  the 
edge  is  traced  until  lost. 

After  the  boundaries  in  a  picture  are  found,  objects 

f  21 
are  to  be  extracted.   Guzman^  J    did  the  work  on  extracting 
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three-dimensional  objects.   The  main  idea  used  to  extract 
the  three-dimensional  objects  was  based  on  the  a  priori 
knowledge  of  the  possibility  of  two  faces  belonging  to  an 
object.   A  vertex  is  in  general  a  point  of  intersection  of 
two  or  more  boundaries  of  regions.   A  program  SEE  has  been 
built  to  examine  the  configuration  of  lines  meeting  at  the 
vertex  to  obtain  evidence  relevant  to  whether  the  .regions 
involved  belong  to  some  object.   Two  types  of  links,  strong 
links  and  weak  links,  are  used.   Figure  1.6  shows  the 
linkage  of  faces  at  several  vertices.   A  solid  line  implies 
a  strong  link  and  a  dotted  line  implies  a  weak  link.   Fig- 
ure 1.7  shows  the  links  which  are  inhibited. 

A  region  is  defined  as  a  surface  bounded  by  simply 
closed  curves.   A  nucleus  is  a  set  of  regions.   Two  nuclei, 
A  and  B,  are  linked  if  the  regions  a  and  b  are  linked  where 
a  e  A  and  b  e  B.   Three  rules  are  set  to  link  the  nuclei. 
First  rule :    If  two  nuclei  are  linked  by  two  or  more 

strong  links,  they  are  merged  into  a  larger 

nucleus . 
Second  rule:   If  nuclei  A  and  B  are  joined  by  a  strong  and 

a  weak  link,  they  are  merged  into  a  new 

nucleus . 
Third  rule :    If  nucleus  A  consists  of  a  single  region,  has 

one  link  with  nucleus  B  and  no  links  with  any 

other  nucleus,  A  and  B  are  merged. 
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"Fork" 


"Arrow" 


Leg"      "Matching  T's" 


Figure  1.6.   Linkage  among  faces 


Figure  1.7.   Several  inhibited  links 
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The  first  rule  is  applied  on  the  picture  repeatedly  until 
it  is  no  longer  possible  to  combine  nuclei.   The  second  and 
third  rules  are  then  applied  successively. 

Figure  1.8  is  an  example  of  the  decomposition  of  three- 
dimensional  objects.   In  step  1,  every  nucleus  corresponds 
to  a  region;  for  example,  nuclei  A,  B  and  C  correspond  to 
regions  a,  b  and  c,  respectively.   There  are  two  strong 
links  connecting  nuclei  A  and  B.   One  link  comes  from  the  Y 
intersection  of  regions  a,  b  and  c.   The  other  link  comes 
from  the  arrow  intersection  of  regions  a  and  b .   All  other 
links  are  derived  in  the  same  way.   Step  2  is  the  straight- 
forward application  of  the  three  rules  to  combine  nuclei. 
The  regions  correspond  to  nuclei  in  a  group   on   an  object, 
for  example,  regions  a,  b  and  c  form  an  object.   It  is  ob- 
vious that  Guzman's  method  can  be  applied  only  to  the  pic- 
tures of  three-dimensional  objects. 

The  second  method  uses  thresholds  to  transfer  a  multi- 
level picture  and  then  finds  the  contours  of  the  binary 
pictures  as  object  boundaries.   Prewitt^11^  used  the  local 
minima  of  the  optical  density^f requency  distribution  of  a 
picture  as  the  thresholds  to  find  the  background  levels, 
cytoplasm  levels  and  nucleus  levels. 

The  third  method  is  the  clustering  method.   Zahnfl6) 
proposed  a  method  to  group  points  into  objects  by  the 
clustering  method,  which  is  graph  theory  oriented.   This 
clustering  method  is  motivated  by  the  perception  of  two- 
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Figure  1.8.   An  example  of  the  decomposition  of  three 
dimensional  objects. 
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dimensional  point  sets  as  separate  "gestalts."   The  princi- 
ple of  grouping  used  is  "proximity"  as  described  by  Wert- 

f  171 
heimer. v     The  proposed  method  is  applicable  to  binary 

pictures.   For  a  binary  picture,  every  picture  point  with 

grey  value  1  is  a  vertex.   Picture  points  having  grey 

values  1  are  called  object  points.   The  connection  between 

object  points  is  called  an  edge.   A  weight  is  assigned  to 

every  edge.   It  is  equal  to  the  Euclidean  distance  between 

the  corresponding  object  points.  An  MST  (Minimal  Spanning 

Tree)  T  is  defined  as  a  spanning  tree  of  G  whose  weight  is 

minimum  among  all  spanning  trees  of  G.   Some  edges  in  the 

MST  can  be  deleted  by  using  a  factor  as  the  measure  of  the 

significant  edge  inconsistency.   The  MST  is   then  clustered 

to  a  forest.   Every  tree  in  the  forest  clusters  together  all 

the  points  in  one  object. 

1.1.3.   Feature  Extraction 

Feature  extraction  strongly  depends  on  the  type  of 
pictures  handled.   How  big  the  feature  set  should  be  de- 
pends on  the  purpose  of  handling  the  picture. 

There  are  two  main  types  of  features.   One  is  the  local 
feature  which  depends  on  individual  objects  in  the  picture. 
Area  and  centroid  were  presented  by  Freeman.    J   Eden^  ' 
has  proposed  the  fundamental  strokes  as  the  features  of 


handwritten  English  characters.   Topological  features  are 
proposed  by  Tcu  and  Gonzalez^  *    for  characterizing  hand- 
written characters.   Topological  features  have  been  used 
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for  automatic  fingerprint   interpretation.  '      A  skeleton^  ' 
has  been  proposed  to  describe  indirectly  the  shape  of  ob- 
jects.  A  skeleton  can  be  thought  of  as  a  generalized  axis 

of  symmetry  of  an  object.   At  first  the  concept  was  applied 

f  201  f  211 

to  the  binary  pictures.   Rosenfeld,  "  ;  Montanari,    }    Phil- 

(22) 
brick^   -*  and  others  have  developed  algorithms  to  find  the 

SKeletons  in  binary  pictures.   Levi v  J    generalized  the 
concept  to  the  multi-level  pictures  by  defining  a  new  dis- 
tance function  which  took  the  grey  level  intensities  into 
consideration.   Ledley^  '    used  the  ratio  of  the  number  of 
concavities  to  the  number  of  segments  of  the  boundary  as 
the  only  feature  in  detecting  the  mitotic  cells. 

The  other  type  of  feature  is  the  global  feature. 
Global  features  are  the  ones  which  reveal  the  interrelation- 
ship among  objects  in  the  picture.   Sometimes  an  object  can 
be  described  in  terms  of  fundamental  components.   Global 
features  can  also  be  used  to  describe  the  interrelationship 

among  fundamental  components  of  an  object.   Inclusion  rela- 

r  2  41 
tion  among  regions  can  be  found  by  MANS.       Several  lin- 
guistic descriptions  have  been  used  to  describe  the  global 

(7) 
features.   Narasimhanv  *    used  syntax-directed  hierarchy 

labeling  to  describe  the  particle  tracks  in  the  bubble 

f251 

chamber.   Shaw  proposed  a  PDL^~  ■*  (Picture  Description 

Language)  which  may  be  the  most  formal  and  useful  linguistic 
approach  in  picture  processing  up  to  date.   Linguistic  ap- 
proach has  the  advantage  of  describing  the  picture  formally. 
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The  PDL  (Picture  Description  Language)  is  a  picture 
or  graph  algebra  over  the  set  of  primitive  structural  de- 
scriptions under  the  operations  +,  -,  x,  *,  -  and  /.   Fig- 
ure 1.9  shows  the  local  completeness  of  the  operations 
[+,  -,  x,  *].   Elements  in  the  PDL  are  considered  equal  if 
they  are  equivalent.   The  equivalent  relation  is  defined  as: 

1.  S.^  is  weakly  equivalent  to  S2  if  there  exists  an  iso- 
morphism between  graphs  of  S,  and  S2  such  that  the  corres- 
ponding edges  have  identical  names. 

2.  Sj  is  equivalent  to  S0    if  (a)  S,  is  weakly  equivalent  to 
S2>  and  (b)  tailCS^  =  tail(S2)  and  head(S;[)  =  head(S2). 

A  number  of  useful  algebraic  properties  are  given  below: 

1.  Each  of  the  binary  operators  is  associative. 

2.  *  is  the  only  commutative  operator;  x  and  -  are  "weakly" 
commutative . 

3.  The  unary  operator  ~ 

(a)  -  acts  as  complementation  in  a  Boolean  algebra. 

(~(s1+s2))  =  CC~s2)+(~s1)3 
(~(s1*s2))  =  CC~s2)*(~s1)) 

(b)  -  obeys  a  "de  Morgan's  law"  with  respect  to  x  and  -. 

(~cs1xs2))  =  ((~s2)-(~s1)) 
C~(s1-s2))  =  ((~s2)x(-s1)) 

(c)  Involution: 
(~C~S))  =  S. 
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Concatenation 


Description 


a+b 


axb 


a-b 


b  +  a 


a*b 


(a+b)*  A 


Figure  1.9.   Local  completeness  of  {+,  x,  -,  *} 


4.  The  /  operator. 

(a)  (/(/s))  =  S. 

(b)  C/(S10bS2))  =  (/S1)j2(b(/S2),  where  0b  is  a  binary 
operator . 

5.  The  null  point  primitive  X. 

(a)  S  +  X  =    X    +  S. 

(b)  S  +  X    =    S,S-X  =  S,  XxS  =  S. 

(c)  ~A  =  X. 

(d)  X0bX  =  X. 

By  using  some  of  the  algebraic  properties  of  PDL  to 
move  unary  operators  and  label  designators  as  far  as  pos- 
sible within  an  expression,  a  standard  form  f(S)  PDL  of  an 
expression  S  can  be  obtained.   f(S)  is  defined  by: 

if  (S  =  s1  V  S  =  C/Sj)  V  S  =  OS^  V  S 

■  (-(/Sj)))  primitive  (S^ ,  then  f(S)  =  S 
else 

if  S  =  (S10bS2),  0be{+,x,-,*},  then  f(S)  =  (f (S^ 0fef (S2) ) 

else 

if  S  =  S1,  then  f(S)  =  f(g(S)) 

else 

if  S  =  (-(S10S2)),  0e{+,*},  then  f(S)  =  (f  (  (~S2) )  0f  (  (-S^  ) ) 
else 

if  S  =  (-(SjXS^),  then  f(S)  =  (f  (  (~S2) ) -f  ( (-S^  ) ) 

else 

if  S  =  C-CSj-S^),  then  f(S)  =  (f  ( (~s2) )  xf  ( (-S^  ) ) 
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else 

if  S  =  (/(S10S2)),  0e{+,-,x,*},  then 

f(S)  =  (f((/S1))0f((/S2))) 
else 

if  S  =  C~C~S.jP),  then  f(S)  =  ffS^ 
else 

if  S  =  C~C/S1))  V  S  =  C/C-Sj)),  then  f(S)  =  f  ( (~f  ( (/S^ ) )  ) 
else 
if  S  =  C/C/S^)),  then  f(S)  =  fCC/Sp) 

A  valid  PDL  expression  (vPDL)  is  the  one  whose  stan- 

dard  form  is  such  that  if  (/p  )  appears  in  it  one  or  more 

£ 
times  for  some  primitive  p  and  label  £ ,    then  p   also  appears 

once  and  only  once  outside  the  scope  of  a  /. 

The  graph  described  by  a  vPDL  S  is  defined  by  the  fol- 
lowing algorithm: 

1.  Transform  S  into  standard  form  by  applying  the  function  f 

£ 

2.  Replace  each  expression  of  the  form  (/p  )  by  a  new  primi- 

tive  p/  .   This  removes  all  /  operators. 

3.  Generate  the  connectivity  graph  of  the  resulting  ex- 
pression. 

4.  Connect  the  tail  and  head  nodes  of  each  edge  p/   to  the 

£ 
corresponding  nodes  of  p  . 

£ 

5.  Eliminate  all  edges  of  the  form  p/  . 

The  above  algorithm  formally  defines  the  meaning  of  labeled 
expressions  and  the  /  operator.   Figure  1.10  shows  the  graph 
of  a  vPDL. 
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step  2 


((((a1  +  b)  *  (b  +  a))*  c)  +  (/a1)) 
((((a1  +  b)  *  (b  +  a))*  c)  +  a1/) 


step  3 


step  4 


step  5 


Figure  1.10.   The  graph  of  a  vPDL, 


21 


It  has  been  proved  that  any  vPDL  describes  a  unique 
primitive  connectivity  and  any  connected  set  of  primitives 
can  be  effectively  described  by  a  vPDL.   It  has  also  been 
shown  that  the  origin  (tail)  of  a  picture  can  be  at  any 
convenient  place. 

The  set  of  rules  or  grammar  G  that  generates  (describes) 
the  class  of  pictures  P„  will  be  a  type  2  (context-free) 
phrase  structure  grammar  with  the  following  restrictions. 
Each  production  is  of  the  form: 

S  ■+  pdl1|pdl2|pdl3|  ...  |pdln,  n  >  1, 
where  S  is  a  non- terminal  symbol  and  pdl.  is  any  PDL  expres- 
sion with  the  addition  that  non-terminal  symbols  are  allow- 
able replacements  for  primitive  class  names.   Sentences  of 
L(G)  will  consist  of  PDL  expressions;  thus,  the  class  of 
terminal  symbols  of  G  will  be  a  subset  of 

{+ ,x, -,*, ~ ,/,(,) }  V{primitive  class  names}  V{label  designators} 

Each  grammar  G  will  have  one  distinguished  non- terminal 
symbol  from  which  L(G)  may  be  generated;  the  symbol  on  the 
left  part  of  the  first  production  of  G  will  be  the  distin- 
guished symbol. 

The  hierarchic  structural  description  H„(C)  of  a  pic- 
ture CePp  having  primitive  structural  description  To(C)eL(G) 
is  defined  as  the  parse  of  T<-.(C)  according  to  G;  H~(C)  is 
conveniently  represented  as  a  parenthesis- free   tree.   A 
simple  example  of  PDL  description  of  a  house  is  given  in 


G: 
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(a) 


House  -*>  C(vm  +  (h+  (»vm) ) )  *  Triangle) 

Triangle   ■>    ((dp    +    dm)    *   h) 

L(G)    =[((vm   +    (h+(-vm)))    *    ((dp   +   dm)*   h))] 


dp  //>  dm\  h 

(a)    G,L(G),    and  primitives 


vm 


Cb) 


A 


Ts(ci)  =  ((vm  +  (h+£vm)))*  ((dp  +  dm)  *  h)) 


H  (c)  : 
s  v  iJ 


House 


(b)  Examples  and  parse  of  a  "house' 


Figure  1.11.   An  example  of  the  structure  descriptions 
of  a  picture. 
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Figure  1.11.   Note  that  all  three  pictures  of  houses  in 
Figure  1.11  have  the  same  primitive  structural  descriptions, 
which  can  be  accepted  by  the  grammar  G. 

PDL  can  describe  very  well  the  interrelationship  be- 
tween primitives,  but  it  does  not  have  the  ability  to  find 
the  primitives.   Figure  1.12  shows  three  pictures  of  chromo- 
somes and  the  accompanying  grammar  to  describe  the  pictures. 


Chromosome  ■*  Kl  *  K2 
K2  ->  v  +  Kl  +  vl  v  +  Kl|  Kl  +  vl  Kl 
Kl  ->■  p+v+p 
primitives:  P  /^      v  aj 


a/1 


T(CX) 

T(C2) 

T(C3) 

'2  3 

(p+v+p) * (v+p+v+p+v) 

(p+v+p) *  (p+v+p+v) 
(p+v+p)* (p+v+p) 


Figure  1.12.   Grammar  for  the  class  of  pictures  of  chromo 
somes  and  three  examples  of  chromosomes. 


It  is  seen  from  Figure  1.12  that  the  PDL  sentences,  which 
describe  human  chromosomes,  are  very  simple  expressions. 
The  main  problem  in  handling  the  pictures  of  chromosomes  is 
to  find  the  primitives. 
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1.1.4.   Classification 

Once  a  good  set  of  features  has  been  extracted,  many 

f  26) 
classification  techniques^  *    are  available.   If  the  set 

of  features  of  different  categories  are  linearly  separable, 

linear  classification^  ''    can  be  used;  otherwise  non-linear 

r  ?6~) 
classification^"  *    should  be  used.   Multi-level  classifica- 
tion^ J    is  sometimes  used.   For  linguistic  description  of 

f  2  5  2  8) 
features,   a  grammar^   '  J    can  be  designed  to  accept  a 

sentence  S  only  if  S  describes  a  picture  of  some  specific 
category.   A  grammar  can  then  serve  the  purpose  of  classi- 
fying objects.   For  example,  any  picture  having  a  PDL  ex- 
pression which  is  acceptable  by  the  grammar  shown  in  Figure 
1.12  is  classified  as  a  chromosome.   Note  that  all  three 
pictures  shown  in  Figure  1.12  will  be  accepted  as  chromo- 
somes.  The  main  problem  of  designing  a  grammar  is  that 
it  has  to  be  complete  in  the  sense  that  it  should  be  able 
to  accept  all  pictures  in  a  category.   Here  we  like  to  em- 
phasize that  in  order  to  have  a  good  result  on  the  classifi- 
cation, a  good  set  of  features  is  required.   If  the  fea- 
tures set  is   poor,  no  matter  how  good  the  classification 
technique  is  the  result  will  be  of  poor  quality. 


1.2.   Summary  of  the  Remaining  Chapters 

Chapters  II  and  III  present  two  different  methods  of 
object  extraction.   Chapter  II  uses  the  gradient  method  to 
find  the  enhanced  picture.   Boundary  points  are  found  by 
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adaptively  thresholding  the  gradients.   Boundary  paths  will 
be  found  in  the  process  of  finding  the  boundary  points. 
Gaps  will  then  be  filled  in  and  the  boundary  segments  are 
then  found.   Special  laws  are  used  to  combine  the  boundary 
segments  to  form  the  boundaries  of  individual  objects. 
Overlapping,  self-folding  and  touching  objects  are  decom- 
posed.  The  gradients  used  are  integers  rather  than  real 
numbers,  such  as  those  used  by  Roberts.  '      Hence  less 
storage  is  required.   The  threshold  is  adaptive  rather  than 
fixed.   It  is  then  less  sensitive  to  the  noise.   Chapter 
III  presents  the  contour  analysis  method.   This  method  was 
motivated  in  experimenting  with  area  picture  data  by  show- 
ing different  levels  of  a  picture  in  a  display  unit.   The 
main  idea  is  that  in  a  small  window  section  in  the  picture, 
the  threshold  for  transferring  the  picture  to  a  binary  pic- 
ture is  approximately  constant.   The  thresholds  are  adap- 
tive rather  than  fixed,  such  as  those  used  by  Prewitt.   The 
result  of  this  method  is  very  successful,  especially  for 
the  area  pictures.   Chapter  IV  discusses  the  graph   theory 
approach  to  the  picture  processing.   Skeletons  of  objects 
can  be  found  by  this  approach.   Further  theoretical  research 
should  be  done  in  this  area. 

Chapter  V  presents  the  extraction  of  several  important 
features  for  area  pictures.   Area,  centroid,  shape,  princi- 
pal axis  direction  and  elongation  index  of  an  object  are 
the  local  features  discussed  in  this  chapter.   Inclusion 
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relation  among  objects  and  the  distribution  of  objects  in  a 
picture  are  the  global  features  presented  in  this  chapter. 
Chapter  VI  discusses  the  experiments  with  biomedical 
images  by  the  above  methods  and  suggests  further  research. 
Chromosome,  skin  cell  and  blood  cell  pictures  are  analyzed. 
In  experimenting  with  chromosome  pictures,  the  main  problem 
is  finding  chromosomes  in  a  picture.   In  experimenting  with 
skin  cell  pictures,  the  main  problem  is  to  detect  the  tumors 
In  experimenting  with  blood  cell  pictures,  the  main  problem 
is  to  find  the  histogram  of  the  intensities.   All  the  ex- 
periments show  very  promising  results.   It  is  hoped  that 
further  research  can  produce  a  more  sophisticated  image  ana- 
lyzing system. 


CHAPTER  II 
OBJECT  EXTRACTION  BY  THE  GRADIENT  METHOD 

When  human  beings  look  at  any  scene,  the  impact  infor- 
mation we  get  is  the  shapes  of  the  objects  in  the  scene. 
The  information  revealing  the  shapes  of  the  objects  are  the 
boundaries.   If  the  objects  are  overlapped,  it  is  possible 
to  use  the  information  of  boundaries  and  grey  intensities  to 
decompose  objects.   As  we  can  very  easily  imagine,  one  way 
to  find  the  boundaries  is  to  use  the  fact  that  usually  the 
boundaries  consist  of  those  points  having  very  high  change 
of  intensities  from  their  neighbors.   Using  this  property  to 
find  the  boundaries  is  called  the  gradient  method,  because 
the  change  of  intensities  is  measured  by  the  gradient.   The 
gradient  method  is  very  good  if  there  are  high  contrasts  in 
the  boundaries,  even  if  there  exist  some  nonuniform  distribu- 
tion of  the  intensities  in  the  objects. 

Some  definitions  will  be  introduced  before  getting 
into  the  problem. 

Definition  2.1. --A  point  p.  in  the  picture  plane  I  is 
an  8-neighboring  point  of  the  picture  point  p  in  I  if  and 
only  if  0<d(p,p- ) <2 ,  where  d  is  the  Euclidean  distance  func- 
tion.  In  order  to  make  the  later  discussion  easier,  the 
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8-neighboring  points  of  p  are  labeled  as  shown  in  Figure  2.1, 
The  set  of  all  8-neighboring  points  of  p  is  denoted  as  N(p). 
The  octal  chain  code  which  encodes  the  line  segment  from  p 
to  p.  is  i.   Because  the  i's  values  range  from  0  to  7,  the 
code  is  an  octal  code. 

It  is  obvious  that  any  curve  in  the  digitized  picture 
is  approximated  by  a  sequence  of  line  segments  which  join 
their  points  to  their  8-neighboring  points.   Hence  any  curve 
in  the  digitized  picture  can  be  represented  by  a  sequence  of 
octal  chain  codes  and  the  start  point  of  the  curve.   Let  C 

be  a  curve  represented  by  the  chain  codes  c,  ...  c   and  the 

r  J  In 

start  point  o.   The  reverse  of  the  curve  C  can  then  be  repre- 
sented by  the  sequence  C"   =  (c  -4)R  ...  (c,-4)„  and  the 
start  point  o*  which  is  the  end  point  of  the  curve  C.   Be- 
cause of  the  small  storage  required  to  store  the  chain  code 
and  because  it  is  easy  to  manipulate,  this  technique  is  used 
throughout  the  dissertation  to  encode  the  curve  in  the  digi- 
tized picture. 


Figure  2.1.   The  8-neighboring  points  of  a  picture  point  p 
and  the  octal  chain  codes. 


29 


2.1.   Boundary  Segments  Finding 

Physically,  a  boundary  point  is  a  picture  point  which 
had  a  high  increase  of  intensity  from  its  neighbors.   The 
rate  of  change  of  intensity  from  p.  to  p,  where  p.  is  an  8- 
neighboring  point  of  p,  can  be  evaluated  by  a  differentiator, 
which  is  defined  as  h*(p,pi)  =  [g(p) -g (pi) ] /d (p ,p . ) ,  where 
g  is  the  picture  function  and  d  is  the  Euclidean  distance. 
It  is  easily  seen  from  Figure  2.1  that  d(p,p.)  -  1  if  i  is 
even  and  d(p,p.)  =  /I   if  i  is  odd.   The  h*  function  can  then 
be  redefined  as  h*(p,pi)  =  ¥*  [g(p) -gCp^  ,E(i)  ]  ,  where  E(i) 
=  0  if  i  is  odd  and  E(i)  =  1  if  i  is  even.   The  V*    function 
is  defined  by  the  following  mapping  table. 


gCp)-g(pl) 

Uf* 

-(n-1)            -(n-2) 

.     -1           0    1 

.    n-2                n-1 

E(i)    0 

-(n-l)//7   -(n-2)//7   .. 

.    -1//T  0    1//2"   . 

.     (n-2)//2~   (n-l)//T 

1 

-(n-1)            -(n-2) 

.    -1           0    1 

.    n-2               n-1 

The  situation  of  a  drop  of  intensity  from  p.  to  p  is 
not  under  consideration,  because  this  situation  will  be  con- 
sidered as  an  increase  of  intensity  from  p  to  p..   Hence  we 
can  define  an  h**  function  which  is  equal  to  the  h*  function 
when  the  value  of  h*  is  equal  to  or  greater  than  0. 
The  value  of  h**  is  set  equal  to  0  when  the  value  of  h*  is 
less  than  0.  For  an  n-levcl  picture  there  are  2n-l  possible 
values  of  h**  function  =  {0 , 1 , . . . , 2n-l} .   Because  the  range 
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of  the  h**  function  is  discrete,  there  is  a  one-to-one  map- 
ping h***  from  R  =  range (h**)  onto  Ib ,  where  Ife  is  a  subset 
of  the  integer  numbers,  and  h***  preserves  the  ordering  of 
the  elements.   h***  serves  the  purpose  of  quantizing  the 
range  of  h**.   A  new  function  h  can  then  be  defined  as  the 
composition  of  h***  and  h**,  that  is,  h  =  h***.h**.   This 
function  is  a  measure  of  the  quantized  rate  of  increase  of 
intensities.   We  can  write  the  h  function  in  terms  of  a  ? 
function,  hCp,Pi)  =  ng(p)-g(Pi),E(i)].   For  n  =  8,  the 
function  can  be  represented  by  the  following  mapping  table: 


V 


E(i)      0 
1 


g(p)-g(Pi) 

-7       -6       -5 


■4       -5       -2       -10       12      3      4 


0        0        0        0        0        0        0      0      1      3      5      6         8      10      11 
0        0        0        0        0        0        002479      12      13      14 


The  magnitude  of  the  gradient  at  a  picture  point  p  is 
defined  as  the  maximal  increase  of  intensity  from  the  neigh- 
boring points  to  the  picture  point.   The  following  definition 
is  then  yielded. 

Definition  2.2. --The  gradient  b  (or  sometimes  will  be 
called  b-value)  at  a  picture  point  p  of  the  picture  g  is 
defined  as 


b(p)  =      max     (h(p,p.)}, 
ie[0,l,...,7]      1 


where  the  h  function  is  defined  earlier. 


31 


The  enhanced  picture  of  a  digitized  picture  g  is  a 
digitized  picture  with  the  intensity  at  every  picture  point 
equal  to  the  gradient  at  the  corresponding  picture  point  in 
the  g  picture.   The  enhanced  picture  ofann-level  picture 
is  of  2n-l  level.   It  will  be  seen  later  that  one  extra 
value  is  required  to  identify  the  boundary  points.   It  is 
then  obvious  that  a  k  bit  picture  will  yield  a  k+1  bit  en- 
hanced picture.   The  enhanced  picture  of  Figure  1.4,  which 
is  a  binary  picture  of  numeral  "6",  is  shown  in  Figure  2.2. 

Because  of  the  unavoidable  noise  appearing  in  the  pic- 
ture, there  is  no  way  to  find  the  real  boundaries  in  one 
step.   We  can  break  the  process- of  finding  boundaries  into 
several  steps.   The  first  step  is  to  find  all  those  points 
which  can  quite  possibly  be  boundary  points.   These  points 
are  called  tentative  boundary  points.   The  fact  that  the 
boundary  points  of  a  boundary  path  are  connected  can  be  used 
in  the  process  of  finding  tentative  boundaries. 

The  following  section  details  the  scheme  of  finding 
the  tentative  boundary  paths.   Octal  chain  codes  are  used 
to  encode  the  paths. 

2.1.1.   Tentative  Boundary  Path  Searching 

The  method  of  finding  the  tentative  boundary  paths  is 
based  on  the  principle  that  a  tentative  boundary  point  is  a 
point  which  has  a  gradient  greater  than  the  gradients  of 
the  neighboring  nonboundary  points  and  is  connected  to  some 
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other  tentative  boundary  points.   Figure  2.3  shows  the  flow 
chart  used  to  search  the  tentative  boundary  paths. 

The  input  to  this  tentative  boundary  path  finder  is 
the  raw  digitized  picture.   The  output  is: 

1.  The  tentative  boundary  uicture  which  is  a  binary  picture 
having  value  -1  at  the  tentative  boundary  point  and  0  else- 
where . 

2.  A  list  of  tentative  boundary  paths.   For  each  tentative 
boundary  path,  it  has 

2a.   the  start  point  of  the  tentative  boundary  path, 

2b.   the  length  of  the  tentative  boundary  path, 

2c.   a  sequence  of  octal  chain  codes  which  encode  the 

path , 
2d.   an  indicator  which  denotes  whether  the  tentative 

boundary  path  is  closed  or  open,  and 
2e.   the  end  point  of  the  tentative  boundary  path  if 
the  indicator  denotes  that  it  is  an  open  path. 
The  threshold  6  in  the  flow  chart,  which  is  used  to 
pick  up  the  first  point  in  a  tentative  boundary  path,  is 
usually  decided  by  the  following  method. 

First,  find  the  histogram  of  the  b  values  in  the  pic- 
ture, which  is  a  plot  of  number  of  picture  points  whose 
gradients  are  greater  than  or  equal  to  a  b-value.   Figure 
2.4  is  the  histogram  of  the  enhanced  picture  shown  in  Fig- 
ure 2.3.   To  determine  which  is  6  from  this  histogram,  find 
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Figure    2.2.      The    enhanced  picture    of  Figure    1.4. 
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Input 

digitized  picture  g 


|  Find  the  enhanced  picture"] 


Note  p*  as 

the 

point 

having 

the 

maximal 

b- 

I'alue  among 

(t>}UN(p)  as 

th 

s  start  point. 

Set 

b(p*)  = 

-1 

and  k 

=  0. 

Note  the  point  p**  such  that  it 
has  maximal  b-value  anong  N(p*) 


Note  p*  as 
the  end  point 


i 


Add  in  the  octal 
chain  code  from 
p**  to  the  start 
point  in  the 
sequence . 


Set  b(p**)  =  -1  and  k  =  k+1. 
Add  in  the  kth  element  of  the 
sequence  as  the  octal  chain 
code  from  p-  to  p**. 


k  = 


yes 


Set  b(p***)=0lp***EX(pWp*}-{p**} 


Set 

p=p*  and 
p*=p*  * . 


Figure  2.3.   Flow  diagram  for  searching  for  tentative 
boundary  paths. 
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the  greatest  drop  in  the  histogram  from  0-1  to 
example  G  will  be  set  as  1. 


In  this 


n, 

600 

* 

400 

- 

200 

* 

* 

0 

• 

i 

12b 
Figure  2.4.   Histogram  of  b-values  of  Figure  2.3 


Searching  the  tentative  boundary  paths  of  Figure  2.2 
yields  two  tentative  boundary  paths: 
path  1 : 

start  point  =  (2,9), 

length  =  52, 

octal  chain  codes  =  4422212122123444434322222101000 
707776676665665656565, 

closed  contour. 
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path  2 : 

start  point  =  (16,10), 

length  =  10  , 

octal  chain  codes  ■  4432100765, 

closed  contour. 

In  this  special  example,  the  tentative  boundary  paths 
are  the  final  boundary  contours.   In  most  of  the  practical 
cases,  because  of  the  existence  of  noise,  overlapping,  self- 
folding  and  touching,  there  will  exist  gaps  between  tentative 
boundary  paths   and   tails  of  tentative  boundary  paths.   The 
following  sections  discuss  the  strategies  of  solving  these 
problems . 

2.1.2.   Procedure  of  Filling  Gaps  and 
Determining  Line  Segments 

To  fill  the  gaps  between  tentative  boundary  paths,  one 

has  to  examine  the  extreme  points  of  tentative  boundary 

paths.   For  an  extreme  point  p  of  a  tentative  boundary  path 

C,  let  p'  be  the  nearest  tentative  boundary  point  which  is 

not  on  C  or  is  on  C  and  has  more  than  five  points  from  p 

along  C.   If  the  Euclidean  distance  between  p  and  p'  is  less 

than  3,  p  is  then  connected  to  p'  through  the  shortest  path. 

If  p'  is  an  extreme  point  of  a  tentative  boundary  path  C , 

tentative  boundaries  C  and  C  will  then  be  combined.   If  p' 

is  not  an  extreme  point,  p'  will  then  be  an  intersection 

node.   Figure  2.5  shows  the  flow  diagram  for  filling  gaps. 
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Input 

the   enhanced  picture    and   the 

list    of   tentative   boundary  paths 


Figure    2.5.      Flow   diagram    for   filling   gaps   between 
tentative   boundary  paths. 
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A  boundary  segment  is  defined  as  a  boundary  path  be- 
tween successive  junction  nodes  or  a  boundary  path  which 
does  not  have  junction  nodes  on  it.   A  tail  of  a  boundary 
is  the  boundary  path  between  an  extreme  point,  which  is  not 
a  junction  node,  and  a  junction  node.   Hence  the  main  pro- 
cedure in  determining  boundary  segments  is  to  order  the 
junction  nodes  along  the  boundary  paths.   Once  boundary 
segments  are  determined,  the  boundary  segments  joined  at  a 
junction  node  can  easily  be  noted.   Because  only  contours 
of  objects  are  of  interest,  all  tails  will  be  erased. 


2.2.   Combining  Boundary  Segments  to  Form 
the  Boundary  Contours  of  Obiects 


Sometimes  a  point  p  on  the  boundary  will  degenerate 
into  several  junction  nodes  after  applying  the  process 
stated  in  the  previous  section  to  the  digitized  pictures. 
It  happens  most  often  when  p  is  a  real  junction  point.   Let 
p  be  a  boundary  point  which  degenerates  into  k  junction 
nodes  n..,...,n,.   N   =  {n,  ,  ...,n,  }  is  the  complete  set  of 
junction  nodes  degenerated  from  the  point  p.   The  ideal 
cases  (i.e.,  no  degeneration)  are: 

1.  point  p  is  not  a  junction  point  and  N   is  empty,  and 

2.  point  p  is  a  junction  point  and  the  cardinal  number 

of  N   is  1. 
P 

Figure  2.6  shows  some  degeneration  cases.   In  the  de- 
generation cases,  there  must  exist  singular  paths  connectinj 
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the  degenerated  junction  nodes.   Let  N  be  the  set  of  all 

junction  nodes  in  the  picture.   The  degenerate  relation  R 

is  defined  on  N  such  that  nRn',  where  n,  n'eN,  if  there 

exists  a  sequence  of  intersection  nodes  n,  =  n,n- , . . .  ,n-.  =  n' 

satisfying  the  condition  that  there  is  a  singular  segment 

between  junction  nodes  n.  and  n-  ,,  i  =  l,...,k-l.   The  de- 

i      l+l 

generate  relation  R  is  obviously  an  equivalence  relation. 
The  equivalence  relation  R  can  thus  partition  N  into  a  col- 
lection of  equivalence  classes.   Every  equivalence  class 
is  then  a  complete  set  of  junction  nodes  degenerated  from 
some  point.   In  the  processing,  we  have  to  find  the  singular 
segments  first,  then  \<e    can  decide  which  junction  nodes  form 
a  complete  set.   As  one  would  expect,  the  most  reasonable 
and  easiest  way  to  determine  if  a  boundary  segment  is  singu- 
lar is  by  the  length  of  the  segment. 


(a) 


ideal 


(b) 


degenerate 


Figure  2.6.   Examples  of  degeneration  from  (a)  a  nonj unction 
boundary  point  and  (b)  a  junction  point  on  the 
boundary. 
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A  threshold  y  is  assigned  such  that  if  a  boundary  seg- 
ment is  of  length  less  than  u,  it  is  classified  as  a  singu- 
lar segment;  otherwise  it  is  nonsingular.   Let  N  be  an  equi- 
valence class  induced  by  the  relation  R  and  let  S  be  the 
set  of  all  singular  segments  connecting  to  the  junction 
nodes  in  N.   (N,S)  forms  a  junction  area.   Let  E  be  the  set 
of  all  nonsingular  segments  connecting  to  the  junction  nodes 
in  N.   The  ordering  of  the  elements  in  E  is  very  useful  in 
combining  boundary  segments. 

It  is  important  to  point  out  that  only  the  chain  codes, 
which  encode  the  line  segments  connecting  the  junction  nodes, 
are  used  to  detect  the  ordering  of  the  nonsingular  boundary 
segments  around  the  junction  area.   The  ordering  of  segments 
around  the  junction  area  can  be  either  clockwise  or  counter- 
clockwise.  Because  of  the  line  encoding  scheme  (octal  chain 
code)  we  used,  there  are  at  most  eight  boundary  segments 
joining  at  a  junction  node.   Figure  2.7  shows  the  flow  dia- 
gram for  ordering  the  nonsingular  segments  around  junction 
areas . 

Every  equivalence  class  of  junction  nodes  can  be 
thought  of  as  a  single  junction  node.   Any  nonsingular  seg- 
ment which  connects  to  some  other  nonsingular  segment  through 
the  equivalence  class  of  junction  nodes  is  considered  as 
through  the  corresponding  single  junction  node.   We  can  then 
imagine  the  boundary  segments  picture  as  an  ideal  one  in  the 
sense  that  no  singular  path  exists. 
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Input 

list  of  boundary  segments 

and  list  of  junction  nodes 


-J  Return  ) 


Note  p  as  a  junction 
node  having  not  been 
processed  yet . 


Set  k  =  0. 

Note  S  is  the  nonsingu 

lar  path  connecting  to  p 


yes 


Set   S   =   S'. 

p  =  extreme  point 
of  S  which  is 
not  p.  


Figure    2.7.       Flow   diagram   for   ordering      nonsingular  paths 
around   junction    areas. 
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A  nonsingular  segment  S.  is  connected  to  a  nonsingular 
segment  S.  if  and  only  if  there  exists  an  equivalence  class 
N  of  junction  nodes  such  that  both  S-  and  S.  contain  some 
junction  nodes  in  N.   This  relation  is  denoted  by  E.   The 
relation  E"  of  connectivity  of  two  nonsingular  paths  is  a 
transitive  closure  of  the  relation  E.   The  picture  consisting 
only  of  the  nonsingular  segments  (and  the  singular  paths 
which  connect  them)  of  the  equivalence  class  induced  by  the 
relation  E'  is  called  an  isolated  picture. 

A  contour,  is  defined  as  a  simply  closed  curve.   We  can 
partition  a  contour  into  a  sequence  of  successive  adjacent 
boundary  segments.   Hence  the  concatenation  of  successive 
adjacent  boundary  segments  can  form  a  contour,  if  the  con- 
dition of  being  a  contour  is  satisfied.   A  picture  point  p 
is  included  by  a  contour  if  p  is  a  point  on  the  contour  or 
if  every  ray  initiated  from  p  will  meet  an  odd  number  of 
times  with  the  contour.   If  the  above  condition  is  not  satis- 
fied, the  picture  point  is  said  to  be  excluded  by  the  con- 
tour.  The  set  of  all  picture  points  which  are  included  by 
a  contour  is  called  the  region  enclosed  by  the  contour. 

Definition  2. 3. --In  an  isolated  picture,  an  elementary 
region  is  defined  as  a  region  enclosed  by  a  minimum  contour 
in  the  sense  that  the  region  does  not  include  any  region 
which  is  enclosed  by  a  contour  in  the  picture.   A  region  in 
an  isolated  picture,  which  is  the  union  of  all  elementary 
regions,  is  called  the  whole  region  of  the  picture.   The 
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contour  in  an  isolated  picture  which  encloses  the  whole 
region  is  called  the  exterior  boundary  of  the  isolated  pic- 
ture.  Any  nonsingular  segment  in  the  exterior  boundary  is 
called  an  exterior  segment. 

Let  m  be  the  number  of  junction  areas  and  k  be  the 

number  of  nonsingular  segments  in  an  isolated  picture; 

f  29) 
there  are  k-m+1  elementary  regions     and  one  exterior 

boundary.   The  above  facts  are  useful  in  terminating  the 

searching  process.   Figure  2.8  shows  the  flow  diagram  for 

finding  all  the  minimum  contours  and  the  exterior  boundary 

in  an  isolated  picture. 

From  here  on,  the  terms  "path"  and  "node"  are  used  to 
imply  "nonsingular  segment"  and  "junction  area,"  respectively, 
unless  otherwise  specified. 

Let  us  look  at  the  different  examples  shown  in  figure 
2.9  to  get  a  feeling  of  "how  our  visual  systems  combine 
paths  into  object  boundary  contours." 

It  is  amazing  that  we  don't  have  to  know  the  intensity 
in  each  elementary  region  to  find  out  that  in  (a)  there  are 
two  objects:   one  is  enclosed  by  the  boundary  contour  con- 
sisting of  paths  Sn  and  S2 ,  the  other  is  enclosed  by  the 
boundary  contour  consisting  of  paths  S-,  and  S,;  in  (b)  there 
is  a  self-folding  object  enclosed  by  the  contour  consisting 
of  paths  Sn  and  S,  with  the  region  enclosed  by  the  contour 
consisting  of  paths  S,  and  S?    as  the  folded  part;  and  in 
(c)  there  are  two  touching  objects:   one  is  enclosed  by  the 
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Input 
isolated 


picture 


Set   k=0,    k'=m-n+l 
and   i=0. 

m=nuTnber   of   paths 
n=nuniber   of   nodes 


Note  p  as  an 

unprocessed  node 


Note  S-:  has  not  combined 
with  Sj+i  (the  index  is 
around  p) . 
Set  S  =  3.j  and  k-k+1. 


I 


i  =  i+1 


Set  S'=Sj+i,  p=the  extreme 
point  of  S'  which  is  not  p, 
Put  S*  in  the  ith  list. 


yes 


A   counterclockwise 
contour    is    the   ex- 
terior boundary. 
A   clockwise    contour 
is   a   smallest 
contour.     


Set    j    =    index   of   S' 
around   the   node   p. 


Figure    2.8.       Flow   diagram   for   finding    all    the    smallest 
contours    and   the    exterior  boundary   in    an 
isolated  picture. 


45 


boundary  contour  consisting  of  paths  S,,  and  S, ,  the  other 
is  enclosed  by  the  boundary  contour  consisting  of  paths  S, 
and  S0. 


(a) 
overlapping 


(b) 
self -folding 


Figure  2.9.   Examples  of  (a)  overlapping,  (b)  self-folding 
and  (c)  touching. 


Sometimes  objects  touching  at  a  point  may  occur  such 
as  the  examples  shown  in  Figure  2.10.   We  call  this  kind  of 
node  the  looping  node. 


Figure  2.10.   Examples  of  looping  nodes. 
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If  there  is  an  odd  number  of  paths  connecting  to  a  junction 
node,  self -folding  or  touching  may  occur.   Usually  the  path 
which  belongs  to  two  objects  (here  we  consider  self-folding 
as  a  special  case  of  touching)  is  quite  "straight."   The 
straightness  of  a  path  can  be  determined  by  the  filtered 
sequence  of  differences  of  successive  chain  codes,  which 
can  be  obtained  by  the  digital  filtering  method.   If  each 
filtered  difference  of  successive  chain  codes  of  a  path 
has  absolute  value  less  than  1,  the  path  is  then  considered 
as  "straight."   Figure  2.11  is  a  flow  diagram  for  combining 
the  paths  in  forming  the  boundary  contours  of  objects. 

Now  we  have  to  find  out  which  objects  are  overlapped, 
which  objects  are  touched  and  which  objects  are  self-folded. 
Let  C,  and  C^  be  the  boundaries  of  two  objects  0-,  and  0~  , 
respectively.   If  there  does  not  exist  a  common  node  between 
C.  and  Cy ,    0,  and  0-  are  separate  objects.   If  there  exists 
a  common  node  between  C,  and  C-  and  if  there  does  not  exist 
a  common  path  between  C,  and  C~,  0,  and  0,,  are  overlapped 
objects,  such  as  Figure  2.9(a).   If  there  exists  a  common 
path  between  C,  and  C~  ,  0,  and  0~  are  either  touching  or 
self -folding .   Assume  that  both  C.  and  C~  have  a  common  path 
S.   Trace  C,  and  C^  so  that  S  is  traced  in  the  same  direc- 
tion.  If  both  the  contours  C,  and  C?  are  traced  in  the 
same  direction  (that  is,  either  both  are  clockwise  or  both 
are  counterclockwise),  0,  and  02  form  a  self-folding  object. 
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Input 

the  list  of  paths  and 

the  list  of  junction  nodes 


1 


Order  paths 
around 
junction  areas 


Find  all  the 
smallest  contours 
and  the  exterior 
boundary  of  every 
isolated  picture. 


0 


Note  an  extreme  point  p  of  a 
path  C  which  has  not  been 
processed  yet. 
Set  C*,  CS=C;  p*,ps=p. 
Count(p')=0  for  all  junction 
nodes  in  the  oicture. 


|count  (p)  =  count  (p)+l| 


©-* 


Erase  C  from  the  list 
Set  C=C*,  p=p*. 


Figure  2.11.   Flow  diagram  for  combining  paths  in  forming 
the  boundary  contours  of  objects. 
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Note  C  =  S-;  and 


C'=S, 


<J+n/2)mod  n" 
(The  ordering  is 
around  p.) 

Erase  C  and  C 
from  the  junc- 
tion p. 


Note  C=Si  and 
C*=S 


<J+1)mod  n 


yes 


yes 


C  is  put  in  the 

list. 

Set  C**CS 

p*«-p, 

C  «-C  ,  and 

p  ■•"extreme  point 
of  C  which  is 
not  p. 


© 


yes 


Note  C  as  the 
first  straight 
path  such  that 
the  paths  between 
C  and  C  around  p 
are  interior  ones 


Erase  C  from  the 
junction  p 


Note  C  which  h; 
p  as  an  extrer.e 
point  and  belongs 
to  a  smallest  con- 
tour containing  C. 
C  has  not  been  in 
the  same  list  as  C 
before. 


Figure  2.11.   Continued 
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Erase  C 
from  the 
junction 
P- 


Erase  C 
from  the 
junction 
P- 


yes 


© 


Note  C  be  the 
first  exterior 
path  connecting 
to  p  such  that 
there  are  in- 
terior paths 
between  them. 


Erase  C  and  C 
from  the  junc- 
tion  p 


Note  C  be  the 
interior  path 
such  that  it 
is  in  the  same 
interior  region 
as  C  and  the 
number  of  paths 
between  C  and 
the  correspond- 
ing nearest  ex- 
terior path  is 
equal  to  the 
number  of  paths 
between  C'  and 
the  correspond- 
ing nearest  ex- 
terior path 


J 


Figure  2.11.   Continued 
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If  C-,  and  C~  are  traced  in  different  directions,  01  and  0? 
are  touching  each  other.   For  example,  in  Figure  2.9(b), 
the  common  path  is  S,  .   Trace  C,  =  S,S-  and  C~  =  S,S~  so 
that  S,  is  traced  upward  in  both  cases.   Both  C,  and  C?  are 
traced  in  the  clockwise  direction.   Hence  CL  and  0?  are 
self-folding.  In  Figure  2.9(c),  trace  C1  =  S-S,  and  C2  =  S-S., 
so  that  the  common  path  S1  is  traced  upward  in  both  contours. 
C,  is  then  traced  in  counterclockwise  direction,  while  C? 
is  traced  in  clockwise  direction.   Hence  C,  and  C?    are  touch- 
ing each  other. 


CHAPTER  III 
OBJECT  EXTRACTION  BY  THE  CONTOUR  ANALYSIS 


It  is  known ^  '    that  boundary  paths  are  very  easily 
extracted  from  a  binary  picture.  Ann-level  picture  g  can 
be  transformed  into  n-1  binary  pictures  g.  =  <j>.  (g)  ,  j=l, 
. . .  .n-1  such  that 


3j(pi)  =  1   if  g(Pi)  >  J 
0    otherwise 

Vt).  e  I,  where  I  is  the  picture  plane  and  j  is  the  threshold 
to  transform  a  grey  picture  into  a  binary  picture.   The 
transformation  from  an  n-level  picture  into  a  collection  of 
(n-1)  binary  pictures  (B-,  ,  •  .  .  ,  3n_1 }  is  denoted  by  $  = 
(<J>,  , .  .  .  ,  <j>   -,}.   The  reason  for  excluding  BQ  pictures  from 
consideration  is  that  it  is  a  trivial  picture  with  l's  every- 
where in  the  picture  plane.   A  binary  picture  B-  is  said  to 
be  a  subpicture  of  a  binary  picture  g.,  denoted  by  g . C  g^, 
if  B-  and  g.  have  the  same  picture  plane  I  and  Vp  e  I, 
B-(p)  =  0  implies  B  •  (p)  =  0.   It  is  easily  seen  that  g-^c 

...c  B   i.   Obviously  |  is  a  one-to-one  transformation  from 
n-1  ' 

an  n-level  picture  into  a  collection  of  (n-1)  binary  pictures 
{B-,,...,B  _1|31c.  ...  c3n_1}.   Figure  3.1  shows  the  binary 
pictures  transformed  from  the  8-level  picture  shown  in 
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Figure    3.1.      The   binary   pictures    transformed   from 
Figure    1.5. 
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(d)    6, 


Figure    3.1.      Continued 
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(e)  B5 


Figure  3.1.   Continued 
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Figure  1.5.   Since  the  highest  level  in  Figure  1.5  is  5, 
both  g6  and  S_  are  trivial  pictures,  having  0  in  every  pic- 
ture point,  and  are  not  shown  in  Figure  3.1. 

Boundary  contours  can  be  extracted  from  $  - ,  j=l,...,n-l. 
Inclusion  relations  can  be  set  up  among  boundary  contours. 
A  boundary  contour  C-.  in  6  -  will  be  included  in  a  boundary 
contour  C?  in  3-,  where  i<j .   If  the  shapes  of  C,  and  C^    are 
similar,  C~    is  more  likely  to  include  the  whole  object.   This 
fact  is  quite  obvious  from  the  experiments.   Using  this  fact 
to  extract  objects  from  the  scene  is  very  effective,  especi- 
ally for  area  pictures,  those  having  only  objects  consisting 
of  areas. 


3.1.   Some  Fundamental  Concepts 
In  Binary  Pictures 

A  picture  point  in  the  picture  plane  of  binary  picture 
B  is  said  to  be  an  object  point  if  B(p)  =  1,  otherwise  it 
is  a  background  point.   A  picture  point  pel  is  said  to  be 
directly  connected  to  p'el  if  d(p,p')<2,  where  d  is  the 
Euclidean  distance  function.   An  object  point  p  is  said  to 
be  connected  to  an  object  point  p*  if  there  exists  a  sequence 
of  object  points  (pQ,...,pm)  such  that  p  =  P0  and  P '  =  Pm 
and  p.  is  directly  connected  to  p^^  i=l,...,m.   A  maximal 
set  of  connected  object  points  in  a  binary  picture  6  is  an 
element  in  that  picture.   An  object  point  p  in  a  binary  pic- 
ture B  is  a  boundary  point  if  there  exists  a  background  point 
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p.  such  that  d(p,p-)<2.   A  contour  is  a  sequence  of  picture 
points  (pQ ,. . .  ,Pm_1)  such  that  0<d (pi  ,Pi  +  1) <2  ,  where  i  «  0, 
...,m-l,p  =Pn  and  m  is  the  length  of  the  contour.   As  stated 
in  the  last  chapter,  it  can  also  be  represented  by  a  sequence 
of  octal  chain  codes.   A  picture  point  p  is  included  by  a 
contour  if  p  is  a  point  in  the  sequence,  or  every  ray  initi- 
ated from  p  will  meet  an  odd  number  of  times  with  the  con- 
tour.  The  set  of  all  picture  points  which  are  included  by 
a  contour  C  is  called  the  .region  enclosed  by  C.   An  exterior 
contour  of  an  element  B  in  a  picture  is  a  contour  such  that 
when  the  contour  is  traced  in  a  clockwise  direction,  then 
all  points  in  the  element  will  be  on  the  right-hand  side. 
An  interior  contour  of  an  element  in  a  picture  is  a  contour 
such  that  when  the  contour  is  traced  in  a  clockwise  direction, 
then  all  picture  points  in  B  are  in  the  left-hand  side.   It 
is  easily  seen  that  for  any  element  B,  there  is  only  one  ex- 
terior contour  and  there  is   a  finite  number  of  interior 
contours.   There  is  one  exterior  contour  and  one  interior 
contour  in  the  binary  picture  shown  in  Figure  1.4. 

Let  B-  be  a  subpicture  of  the  binary  picture  B-.   For 
any  exterior  contour  C^J  in  B-  there  exists  an  exterior 
contour  C^  '    in  B-  such  that  the  region  enclosed  by  C  *•-'''  is 

a  subset  of  the  region  enclosed  by  C^   .   For  any  interior 

fi  V 

contour  CKJJ      in  B-  there  may  exist  an  interior  contour 

fi")  '  fi")  ' 

C^  J      in  B-  such  that  the  region  enclosed  by  C  *■  *       is  a 


subset  of  the  region  enclosed  by  C 


,  rOV 
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3.2.   Contours  Finding  in  Multi-level  Picture 

In  a  multi-level  picture  g,  a  picture  point  p  is  a 
boundary  point  if  there  exists  a  point  p.  such  that  d(p,p.)<2 
and  g(p)>g(p-)>   P-:  is  called  an  adjacent  background  point, 
with  respect  to  the  contours  passing  through  the  boundary 
point  p  in  binary  pictures  S • ,  g(p ■ ) <j <g  (p) .   In  order  to 
find  the  contours,  a  labeling  scheme  is  used.   Let  g  be  a 
n-level  picture.   Assume  that  n  is  even,  which  is  generally 
the  case,  as  n=8  for  the  picture  shown  in  Figure  1.5.   The 
rules  of  the  labeling  scheme  are: 

1.  all  boundary  points  in  the  i    contour  are  labeled  as 
n-l+2i, 

2.  all  adjacent  background  points  with  respect  to  the  i 
contour  are  labeled  as  n-2+2i. 

3.  the  labeling  of  the  odd  numbers  has  priority  over  that 
of  the  even  numbers,  and 

4.  for  odd  numbers,  the  labeling  of  large  numbers  has  prior- 
ity over  the  small  numbers.   For  even  numbers,  the  labeling 
of  small  numbers  has  priority  over  the  large  numbers. 

It  is  easily  seen  that  a  contour  C  in  a  multi-level 

picture  is  a  contour  in  the  binary  pictures  S-,  b(C)<j<g(C), 

where  g(C)  =  min{g(p)}  and  b(C)  =  max{g(q)|q  is  an  adjacent 

peC 
background  point  with  respect  to  C}.   g(C)  and  b(C)  are 

called  the  intensity  and  the  background  intensity  of  the 

contour  C,  respectively. 
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A  rotator  is  a  vector  initiating  from  an  object  point 
to  its  eight-neighboring  points,  and  hence  can  be  represented 
by  an  octal  chain  code.   A  rotator  rotating  in  counterclock- 
wise direction  is  needed  in  the  process  of  finding  contours. 
If  a  rotator  pointed  to  the  picture  point  pif  it  will  next 

point  topr.,,~.       __ 

*■    •'modS'      rotator  is  the  index  of  the  neigh- 
boring point. 

In  finding  the  contours  in  a  picture  g,  first  scan  the 

picture  g  in  forward  direction.   Let  p  be  the  first  picture 

point  satisfying  the  conditions  that  g(p)>g(p  )  and  either  (*   (W 

the  picture  point  p  has  not  been  labeled  before,  or  the 

picture  point  p  is  on  some  contour  C  ,  which  has  been  found, 

and  g(C')<g(p).   p  is  then  the  start  point  of  a  contour  to 

be  found.   Set  the  initial  position  of  the  rotator  in  the 

direction  indicated  by  octal  chain  code  1.   Find  the  first 

object  point  pointed  by  the  rotator  which  rotates  in  a 

counterclockwise  direction.   After  the  second  contour  point 

has  been  found,  the  initial  position  is  set  at  the  direction 

indicated  by  the  octal  chain  code  (i+4)   ,    where  i  is  the 

octal  chain  code  indicating  the  position  of  last  rotator. 

By  this  method  of  finding  the  contours,  an  interior  contour 

will  be  encoded  in  a  clockwise  direction,  while  an  exterior 

contour  will  be  encoded  in  a  counterclockwise  direction. 

Let  c1...cm  be  the  sequence  of  the  octal  chain  codes 

of  a  contour  C.   c.ci+1  is  deleted  if  |ci-c+1|  =  4.   By 

this  procedure,  the  sequence  of  the  chain  code  will  be 
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reduced  to  c^,...c  ,.   If  the  reduced  sequence  is  empty, 

then  the  contour  is  one  which  can  be  broken  into  two  paths 

S,  =  c, ...c    and  S~  =  c   .^...c   such  that  S-  =  S  ~. 
11    m,      l         m.  +i    m  21 

A  contour  having  an  empty  reduced  sequence  is  considered  to 
be  an  exterior  contour.   If  the  reduced  sequence  is  net 
empty,  then  the  sum  of  the  differences  of  adjacent  octal 
chain  code  is  the  parameter  to  indicate  whether  the  contour 
is  encoded  clockwise  or  counterclockwise.   If  the  sum  is  -8, 
the  contour  is  traced  in  a  clockwise  direction.   Hence  the 
contour  is  an  interior  one.   If  the  sum  is  +8,  the  contour 
is  traced  in  a  counterclockwise  direction.   Hence  the  con- 
tour is  an  exterior  one.   Figure  3.2  shows  the  flow  diagram 
used  to  find  contours  in  a  multi-level  picture. 

Figure  3.3  is  the  labeled  picture  of  the  8-level  pic- 
ture shown  in  Figure  1.5. 

There  are  14  contours  in  Figure  1.5. 

Contour  1:   start  point  =  (1,1), 

octal  chain  codes  =  22222222222222224444444444 

4456  56  5666  566656660000  0  000 
000000000, 

length  =  61, 

intensity  =  1, 

background  intensity  =  0, 

exterior  contour. 

Contour  2:   start  point  =  (1,1), 

octal  chain  codes  =  22222222222222224444534454 

5  56  566  56  766  76  5  500  0000  00  000 
000 
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length  =  55 , 

intensity  =  2 , 

background  intensity  =  1, 

exterior  contour. 
Contour  3:   start  point  =  (1,1), 

octal  chain  codes  =  22222222666656660 

length  =  17, 

intensity  =  3, 

background  intensity  =  2, 

exterior  contour. 
Contour  4:   start  point  =  (1,6), 

octal  chain  codes  =  44344540000000 

length  =  14, 

intensity  =  3, 

background  intensity  =  2, 

exterior  contour. 
Contour  5:   start  point  =  (1,10), 

length  =  0, 

intensity  =  4 , 

background  intensity  =  3, 

exterior  contour. 
Contour  6:   start  point  =  (2,1), 

octal  chain  codes  =  22650, 

length  =  5 , 

intensity  =  4, 

background  intensity  =  3, 

exterior  contour. 
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Contour  7:   start  point  =  (2,6), 

octal  chain  codes  =  4444322102111007666656653, 

length  =  25, 

intensity  =  2, 

background  intensity  =  1, 

interior  contour. 
Contour  8:   start  point  =  (5,12), 

octal  chain  codes  =  1210211022134345456655676660, 

length  =  28, 

intensity  =  3, 

background  intensity  =  2, 

exterior  contour. 
Contour  9:   start  point  =  (9,11), 

octal  chain  codes  =  2171222245556660, 

length  =  16  , 

intensity  =  4, 

background  intensity  =  3, 

exterior  contour. 
Contour  10:  start  point  =  (11,4), 

octal  chain  codes  =  2221776553, 

length  =  10, 

intensity  =  2  , 

■ 

background  intensity  =  1, 

interior  contour. 

Contour  11:  start  point  =  (12,6), 

length  =  0 , 


-> 


dUJ. 
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intensity  =  4, 

background  intensity  =  3, 

exterior  contour. 
Contour  12:  start  point  =  (12,10), 

length  ■  0, 

intensity  =  5, 

background  intensity  =  4, 

exterior  contour. 
Contour  13:  start  point  =  (14,1), 

octal  chain  codes  =  222451766, 

length  =  9, 

intensity  =  3, 

background  intensity  =  2, 

exterior  contour. 
Contour  14:  start  point  =  (16,6), 

octal  chain  codes  =  1753, 

length  =  4, 

intensity  =  2, 

background  intensity  =  1, 

interior  contour. 


■■ 
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Note  the  it"  contour 
as  interior  contour, 
if  it  is  clockwise. 
Otherwise  the  i.th 
contour  is  noted  as 
the  exterior  contour. 


Input 
digitized 
picture  g 


Set  start  point  as  p. 
i=i+l,  j=j+2,  k=k+2, 
and  L(p)=j . 

Initialize  the  rotator 
bv  setting  r=0. 


Find  the  first  p_,  satisfyinj 

the  conditions  of  being  a 

boundary  point  and  having 

g(pr)  >§'(£*),  if  C*  exists. 

Set  L(or)=L. 

Store  r  in  the  itn  contour 

sequence . 

Set  p=pr  and  r=fr+4)   ,  „. 
'mod  8 


Label  the  corresponding  adja- 
cent background  points  p's  as 
k  if  L(p)<n. 


~J 


Figure  3.2.   Flew  diagram  for  finding  contours  in  a  multi 
level  picture. 
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13131111 
19191221 
19131221 
19131221 
13122120 
13122120 
13122120 
13122120 
13122120 
11122721 
11272627 
27262627 
27262627 
33272627 
33322722 
33323335 
33331111 


21151 
20212 
20202 
20  1 
20  1 
1  1 
1  12 
1202 
20202 
21212 
22232 
22292 
22232 
23  32 
35232 
34352 
35  91 


51515 

12121 
02020 
111 
12020 
12021 
02021 
02123 
12223 
22325 
32530 
42530 
42530 
425  4 
32525 
22323 
11114 


17151515 
21151414 
20211411 
20212211 
20212323 
2123  323 
2223  323 
23242423 
24252524 
24252524 
25302523 
31302522 
30252311 
25242311 
23231110 
11111010 
9  9  9  9 


1111  9 

411010 

1010  1 
1010  1 
1110  1 
1110  1 
111010 
221110 
2311  9 
2310 
1110 
1110 
10  9 


9 

9 
9 
8 

9  8 
8  8 
8  0 
8  0 


.-K 


8  0 

8  0 

8  0 

8  0 

0  0 

0  0 

8  0  0 

8  0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 


Figure  3.3.   The  labeled  picture  obtained  from  Figure  1.5 
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3.3.   Inclusion  Relation  Among  Contours 

As  stated  in  the  beginning  of  the  chapter,  inclusion 
relation  must  be  set  among  contours  to  extract  objects. 
Inclusion  relation  can  be  easily  found  from  the  labeled  pic- 
ture L  and  the  original  n-level  picture  g.   The  labels  and 
intensities  of  two  successive  picture  points  p   and  p  (in 
the  forward  raster  direction,  i.e.,  p  is  at  the  right-hand 
side  of  the  picture  point  p  )  are  required.   The  information 
from  the  label  of  the  picture  point  p   is  stored  as  a  state. 

Three  kinds  of  labels  exist.   State  1  is  that  L(p  )<n,  that 

_  r  o 

is,  pQ  is  not  labeled  in  the  labeling  process.   State  2  is 
that  L(p0)>n  and  is  even,  that  is,  p   is  an  adjacent  back- 
ground point.   State  3  is  that  L(p  )>n  and  is  odd,  that  is, 
pQ  is  a  boundary  point.   Figure  3.4  is  the  state  diagram 
for  finding  the  inclusion  relation  among  contours.   The  ac- 
tions in  Figure  3.4  should  be  explained.   An  array  is  initi- 
ated every  time  a  line  in  the  picture  is  scanned.   State  1  is 
the  initial  state.   If  the  action  consists  of  entering  the 
region,  the  region  name  is  put  in  the  array.   If  the  action 
consists  of  leaving  the  region,  the  region  name  is  taken 
out  of  the  array.   The  inclusion  relation  is  a  partially 
ordered  relation,  which  can  be  represented  by  a  Hasse 
graph. *•     Every  time  a  contour  C  is  put  in  the  array,  the 
contour  C  is  included  in  the  contour  which  is  next  to  C  in 
the  array.   At  a  picture  point  p,  there  may  pass  several 
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L(p)>n  5  is  even  /  no  action 


L(p)>n  $  is  even 
/  no  action 

Up)<X? 
no   action 


L(p)>n  5  is  odd,  if  the 
contour  at  p  is  an  ex- 
terior contour,  if  g(p0) 
>g(p)  /  no  action,  if 
jfCPo)<g(p)  /  enter  the 

igion,  if  the  contour  at 
p  ^  an  interior  contour, 
if  |^p0)>g(p)  /  enter  the 
regions-  Tf  g(Po)<gCp)  / 
no  actron. 


L(p)<n  / 
no  action 


L(o)>n  §  is  even  if  t\ne 
contour  at  p0  is  an  exv- 
terior  contour,  if  g(p( 
>g(p)  /  leave  the  regioi 
if  g(p0)£g(p)  /  no  actioh; 
if  the  contour  at  p0  is  aW 
interior  contour,  if  g(po)\ 
>g(p)  /  no  action,  if  g(p0) 
<g(p)  /  leave  the  region. 


L(p)>n  §  is  odd  /  enter 
the  region. 


L(p)<n  iiNt^he  con- 
tour at  p0  rs^an 
exterior  contoui 
no  action,  if  the 
contour  at  p0  is  an 
interior  contour  / 
leave  the  region. 


if  the. 

"aTpi  s  an  ex- 
terior contour, 
if  gCpoO  g(p)  / 
enter  the  region, 

if  g(Po)_gCp)  / 
no  action,  if 

g(P0)  §Cp)  / 
leave  the  region, 
enclosed  by  an 
exterior  contour 
passing  p0. 


L(p)>_n  §  is  odd 

if  the  contour  at 
p  is  an  interior 
contour,  if  g(p0) 
^_g(p)  /  enter  the 
region  and  leave 
the  region  enclosed 
by  an  interior  con- 
tour passing  p0, 

if  g(P0)>g(P)  / 
enter  the  region 
and  leave  the  re- 
gion enclosed 
either  by  an  inter- 
ior contour  or  an 
exterior  contour  C, 
with  intensity  g(C) 
>g(p)  passing  p0. 


Figure  3.4.   State  diagram  for  finding  the  inclusion 
relation  among  contours. 
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contours  C, ,. . . ,C  .   Let  C,  .....  ,C  , ,  where  the  indices  are 
1 '    '  m        1     '  m  ' 

ordered  according  to  the  ascending  order  of  the  correspond- 
ing label  values,  be  the  contours  satisfying  the  condition 
of  entering  the  region  at  the  picture  point  p.   C, ,,..., C  , 
will  be  put  in  the  array  in  the  order. 

Applying  this  inclusion  relation  finding  process  to 
the  contours  in  Figure  1.5,  the  Hasse  graph  will  turn  out 
to  be  the  one  shown  in  Figure  3.5. 

3.4.   Object  Extraction  by  Comparison 

Now  we  are  in  the  final  stage  of  extracting  objects. 
Let  H  be  the  Hasse  graph  representing  the  inclusion  relation 
among  contours  in  a  picture.   Let  the  area  of  the  region 
enclosed  by  a  contour  C  be  denoted  as  A  (which  can  be  found 
by  the  method  presented  in  Section  5.1.1.).   A  threshold 
cu  is  set  such  that  if  A  <  a-.  ,  the  node  corresponding  to  C 
is  deleted  from  Hj   It  is  obvious  that  if  a  contour  C  satis- 
fies the  above  condition,  all  its  descendants  will  satisfy 
the  condition  and  will  thus  be  deleted.   A  reasonable  value 
for  a,  would  be  9,  because  it  is  usually  impossible  to 
filter  out  the  noise  disturbing  the  shape  of  a  contour  if 
the  area  enclosed  by  the  contour  is  less  than  9.   Let  H' 
be  the  subgraph  of  H  obtained  by  this  deletion  process. 
The  H'  obtained  from  the  Hasse  graph  H  shown  in  Figure  3.5 
is  shown  in  Figure  3.6. 
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1 


Figure  3.5.   The  Hasse  graph  representing  the  inclusion 
relation  among  contours  in  Figure  1.5. 
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Figure  3.6.   The  Hasse  graph  obtained  through  the  deletion 
of  small  contours. 


A  dissimilarity  measurement  between  two  contours  C, 
and  C0  is  defined  as 

dcCi,c2)  .  Im^-m^Hm^-m^Mh^-m™!  ■ 

where  M^1-' ,  M^1-'  and  M*-1-'  are  the  number  of  critical  points, 
c   '   p        v 

the  number  of  peak  points  and  the  number  of  valley  points 
on  the  contour  C- ,  i  =  l,  2. 

A  threshold  a2  is  set  to  extract  objects  from  H' .   If 
a  node  corresponding  to  a  contour  C  is  the  only  son  of  a 
node  corresponding  to  a  contour  C* ,    and  if  D(C,C*)  <  a^ , 
the  node  corresponding  to  the  contour  C  is  deleted  from  H' . 

First  the  levels  of  the  nodes  in  H*  are  assigned.   The 
level  of  the  root  in  H'  is  assigned  as  1  and  the  levels  of 
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all  sons  or  nodes  of  level  k  are  assigned  as  k+1.   The 
deletion  procedure  is  then  applied  to  H'  from  the  nodes 
with  largest  level  assignment  to  the  nodes  with  level  1. 
Let  the  resulting  graph  be  denoted  as  H".   Every  node  in  H" 
can  possibly  correspond  to  the  contour  of  an  object.   Figure 
3.7  shows  H"  obtained  from  Figure  3.6. 


Figure  3.7.   The  Hasse  graph  obtained  through  the  deletion 
of  similar  contours. 


Contours  7  and  10  are  not  under  consideration  because  they 
are  interior  contours.   Contour  1,  which  touches  the  pic- 
ture frame,  is  also  not  under  consideration.   Hence  only 
one  object,  which  is  enclosed  by  contour  8,  is  extracted. 


CHAPTER  IV 

GRAPH  THEORY  APPROACH  TO 
PICTURE  PROCESSING 


In  this  chapter  we  are  proposing  a  method  to  extract 
objects  in  a  multi-level  picture  by  the  clustering  method. 
This  approach  can  detect  the  gestalt  clusters,  which  are  ob- 
jects, in  the  picture  and  can  presumably  give  the  "skeletons" 
of  objects.   We  first  transfer  the  n-level  picture  into  a 
weighted  graph  G  and  then  f ind  an  MST  (Minimal  Spanning  Tree) 
of  every  isolated  weighted  graph  G.  of  G.   Based  on  the 
statistics  of  an  MST.  we  can  cluster  an  MST.   Every  cluster 
is  an  object  in  the  picture.   Some  major  paths  of  an  MST  re- 
stricted to  a  cluster  form  a  "skeleton"  of  an  object.   Seme 
graph  theory  backgrounds  and  properties  of  the  MST  will  be 
discussed  in  the  following  section. 


(29) 
4.1.   Some  Graph  Theory^     Backgrounds 

and  the  Properties  of  an  MST 

An  undirected  finite  graph  G  =  {V,E,F}  consists  of  a 

set  V  of  m  vertices,  where  V  =  {v, ,...,v  },  a  set  E  of  k 

'  1 '    '  m  ' 

edges,  where  E  =  {e,  ,  .  . .  ,e,  } ,  and  a  function  F,  a  mapping 
from  E  into  V  and  V,  the  set  of  all  unordered  pairs  of  mem- 
bers of  V.   Figure  4.1  shows  an  example  of  an  undirected 
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Figure  4.1.   An  example  of  an  undirected  finite  graph 


graph.   In  Figure  4.1,  V  =  {v-^...^}  and  E  =  {e1,...,eg} 


If  e.  is  in  E,  then  F(e.)  =  (v.  §v.  ),  for  some  vert 


V  x2 


ices 


3  '        v  3 

v.   and  v.   in  V,  such  as  F(e.)  =  (v9§v7)  in  Figure  4.1. 

An  edge  e.  is  incident  with  vertices  v.   and  v.  ,  if  Ffe-1 
3  ix      i2'        jJ 

=  Cv-  qv-  ).   For  example,  in  Figure  4.1,  e,  is  incident 
xl   x2  i 

with  v?  and  v  .   If  F(e.)  =  (v.  §v.  ),  then  e.  is  a  loop, 
^      j         j      i1   i2         j 

such  as  eg  in  Figure  4.1.   The  number   n(v.)  of  edges,  which 

are  incident  with  a  vertex  v. ,  is  called  the  degree  of  the 

vertex  v..   For  example,  n(v?)  =  4  in  Figure  4.1.   v.   and 

i1 

v.   are  adjacent  vertices,  if  there  exists  an  edge  e.  such 

2  3 

that  F(e.)  =  (v.  %v.    ).   For  example,  v.  and  v.  are  adjacent 

J        12  L  £ 

vertices  in  Figure  4.1.   Let  e.   and  e.   be  two  distinct 

Jl  J2 

edges.   If  F(e.  )  =  (v   §v.  ),  and  if  F(e.  )  =  (v.  §v.  ), 

Jl       Xl  12  ^2  x2      x3 

then  e.   and  e.   are  adjacent  edges.   Furthermore,  if 

Jl       J2 
Vj   =  v,  ,  then  e.   and  e.   are  parallel  edges.   For  example, 

3  1        Jl      J2 
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in  Figure  4.1,  e,  and  e,  are  adjacent  edges,  and  e,  and  e? 
are  parallel  edges.   A  simple  graph  is  a  graph  having  no 
loop  and  no  pair  of  parallel  edges.   The  graph,  shown  in 
Figure  4.1,  is  not  a  simple  graph,  because  e.,  and  e~  are 
parallel  edges,  and  e„  is  a  loop.   Figure  4.2  shows  an  ex- 
ample of  a  simple  graph. 


v, 

-o  < 


v3  v5 

Figure  4.2.   An  example  of  a  simple  graph, 


A  graph  G'  =  {V'jE'jF'}  is  a  subgraph  of  a  graph  G  = 
{V,E,F} 

1.  if  V  is  a  subset  of  V  and  E'  is  a  subset  of  E, 

2.  if  for  every  e.  in  E',  F'(e.)  =  F(e.)>  and 

3.  if  for  every  e.  in  E',  F(e.)  =  (v.  $v.  ),  v.   and  v. 

j  j       i1   i2     x1  i2 

are  in  V . 

A  finite  sequence  of  edges,  e.  ,...,e.  ,  is  an  edge 

Jl      Jt 
progression  (or  edge  sequence)  of  length  t  if  there  is  a 

sequence  of  vertices,  v.  ,v.  ,...,v.  ,  such  that  for  each 

x0  xl  xt 

y  =  l,...,t,  F(e.  )  =  (v.    §v.  ).   If  v.  f   v.  ,  the  edge 

Jy      xy-l   xy         0    1t 
progression  is  open  (or  non-cyclic),  such  as  e,,  e.,  e,,  e 

in  Figure  4.1.   If  v.   =  v.  ,  the  edge  progression  is 

10     Xt 
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closed  (or  cyclic),  such  as  e.,,  e,  ,  e? ,  e,  in  Figure  4.1. 
An  edge  progression  is  said  to  be  from  v.   to  v.  ;  v.   is 

the  initial  vertex  and  v-   is  the  terminal  vertex  of  the 

t 
progression.   For  y  =  l,...,t-l,  v.   is  an  intermediate 

V 
vertex  of  the  progression.   A  chain  progression  (or  non- 
cyclic  path)  is  an  open  edge  progression  in  which  no  edge 
is  repeated  in  the  sequence,  such  as  e,,  e~,  e,  in  Figure 
4.1.   A  circuit  progression  (or  cyclic  path)  is  a  closed 
edge  progression  in  which  no  edge  is  repeated  in  the  se- 
quence, such  as  e.,,  e.,  e,,  efi,  e_,  eq  in  Figure  4.1.   A 
simple  chain  progression  (or  simple  path  or  arc)  is  a  chain 
progression  in  which  no  vertex  is  repeated  in  the  vertex  se- 
quence, such  as  e.-,  e?    in  Figure  4.1.   A  simple  circuit 
progression  (or  circuit)  is  a  circuit  progression  in  which 

v.   =  v.   but  there  is  no  other  duplication  of  any  vertex 

10     Xt 
in  the  vertex  sequence,  such  as  e ., ,  e_,  eq  in  Figure  4.1. 

Let  v.   and  v.   be  two  vertices  of  a  graph  G,  v.   and 
10       t  10 

v.   are  connected  vertices  if  v.   =  v.   or  if  there  exists 

Xt  x0     Xt 

an  edge  progression,  e.  ,...,e.   with  vertex  sequence  v.  , 

Jl      Jt  x0 

v.  ,...,v.  .   The  existence  of  an  edge  progression  from 

Xl      Xt 
v.   to  v.   implies  the  existence  of  an  arc  from  v-   to  v.  , 
X0      \  x0  V 

so  a  pair  of  distinct  vertices  is  connected  if  and  only  if 

there  is  an  arc  joining  them.   G  is  a  connected  graph  if 

for  any  vertices  v.  and  v.  in  V,  v.  and  v.  are  connected. 
7  l       j      '   l       3 

A  mxm  matrix  A  =  (a-  )  can  be  defined  such  that 

1j 

(1  if  i=j  or  if  v.  and  v.  are  adjacent  vertices 
0   otherwise. 
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For  example,  the  A  matrix  for  the  graph  shown  in  Figure  4.2 
is 


A  = 


fmool 

11000 
10100 
00011 
00011 


For  some  integer  S  such  that  As  =  AS+  ,  the  matrix  As  = 


(s) 


(s)  _ 


(a|  J)    is  called  the  connection  matrix  in  that  a^J    =  1  if 
and  only  if  the  vertices  v.  and  v.  are  connected.   For  ex- 
ample, the  connection  matrix  of  the  graph  shown  in  Figure 
4.2  is 


'11100' 

A2- 

11100 

11100 

00011 

00011 

The  connectivity  relation  C  on  the  vertices  of  a  graph- 
is  an  equivalence  relation.   Let  the  partition  of  V,  by  the 
connectivity  relation,  be  q  equivalence  classes,     {V, , . . . , 
V  }.   For  example,  V   =  {v1,v7,v.}  and  V-  =  {v,,v5>  in 
Figure  4.1.   Let  E,  be  the  subset  of  E  each  of  which  is 
incident  iv'ith  vertices  in  V..   For  any  i  f   j  ,  there  does 
not  exist  an  edge  e   in  E  such  that  it  joins  a  vertex  in  V. 
and  a  vertex  in  V..   Thus,  if  e^  is  in  E-  and  F(e^)  =  (v.  S 

v.  ),  then  both  v.   and  v-   are  in  V..   Hence  {E,,...,E  } 

12  11  12  1  L  q 

is  a  partition  of  E.   For  example,  E,  =  {e,,e2>  and  E?  = 

{e7}  in  Figure  4.2.   Therefore,  G.  =  (V.,E.,F.),  where  F. 

is  the  restriction  of  F  to  E.,  defines  a  subgraph  of  G. 

Each  such  G-  is  obviously  connected.   If  G.  is  a  subgraph 
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of  a  connected  subgraph  G'  of  G,  then  G.  =  G';  that  is,  G. 
is  a  maximal  connected  subgraph  of  G  and  is  called  an  iso- 
lated component  of  G. 

Definition  4.1. --A  tree  is  a  connected  graph  having  no 
circuit.   A  circuit-free  graph  having  q  connected  components 
is  a  forest  of  q  trees. 

If  T  =  {V,E,F}  is  a  tree  and  e  is  an  edge  of  T,  then 

the  subgraph  G  =  {V,E-(e),  Fp_f  ■>}    of  T  is  disconnected, 

where  F-  ,    -,  implies  the  function  F  restricted  on  the  domain 
E-Cej 

E-(e).   Hence  no  subgraph  derived  from  a  tree,  which  has  all 

the  vertices  and  lesser  number  of  edges, is  connected.   Thus 

a  tree  is  a  minimal  connected  graph. 

Definition  4.2. --Let  G  =  {V,E,F}  be  a  connected  graph, 

and  let  v.  and  v-  be  two  distinct  vertices  in  V.   The  dis- 
i      3 

tance  d(v.,v.)  between  v.  and  v.  is  defined  as  the  minimum 
i'  jJ  i  j 

length  of  the  arcs  from  v.  to  v..  If  v-  =  v.,  dfv.,v.)  is 
defined  equal  to  0. 

The  distance  function  defined  above  satisfies  the 
metric  axioms: 

1.  d(vi,vi]  =  0, 

2.  d(vi,v.)  =  d(v.  ,vi),  and 

3.  d(vi,vt)  <_   dO-^v.)  +  d(v.,vt)  V  vi,v.,vt  e  V. 

Definition  4. 3. --Let  T  =  {V,E,F}  be  a  tree  and  v.  is 

a  vertex  in  T.   If  n(v-)  =  1,  vertex  \.    is  termed  as  a  leaf 

of  the  tree  T.   An  arc  a(v.,v.)  from  v.  to  v.  is  called  a 

i  r       i     J 

diametral  path  when  its  length  I    is  maximal  among  the 
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distances  between  any  two  vertices;  I   is  the  diameter  of 
the  tree  T.   A  vertex  c  in  V  is  a  center  of  T  if 


r (c)  =   min  {r (v. ) }  =  rn 
v±eV  x      U 

where  r(v.)  is  defined  as   max{d (v. , v . ) } .   r„  is  called  the 
i  VjeV    i   3       0 

radius  of  T.   Let  v.  be  a  leaf  of  T.   The  longest  arc  from 
l  to 

v.  is  called  a  major  path  from  v.. 

l  J  l 

The  following  theorem  reveals  the  properties  of  the 
centers  of  a  tree. 

Theorem  4.1. --Let  T  be  a  tree  of  diameter  I    and  afv.  , 

V 

v.  )  be  a  diametral  path,  having  the  corresponding  sequence 

xl 
of  vertices  v.  ,v.  ,...,v.  .   When  I    is  even  T  has  a  single 

10    1       I 
center  c  =  v-       and  has  a  radius  rn  =  1/2.      All  major 

1(£/2)  U 

paths  go  through  c.   When  I    is  odd  T  has  two  centers, 

c,  =  v.         and  c0  =  v.        ,  and  has  a  radius  r~  = 

1     1(i-l)/2       L  1(£+l)/2  U 

(£+l)/2.   All  major  paths  pass  through  both  centers. 

Definition  4. 4. --Let  G  =  {V,E,F}  be  a  connected  graph, 
and  T  =  {V„,ET,FT}  be  a  tree  and  a  subgraph  of  G.   If  V„  = 
V,  then  T  spans  G,   T  is  termed  a  spanning  tree  of  G. 

Definition  4. 5 . --A  weighted  graph  G  =  {V,E,F,W}  is  a 

graph  {V,E,F}  with  the  assignment  of  a  weight  to  each  edge 

in  E.   W  is  the  weight  function  which  maps  E  into  real 

numbers,  that  is,  the  weight  of  an  edge  e.  is  W(e.).   The 

weight  of  an  edge  progression  e.  ,...,e.   is  defined  equal 

31  Jt 

to  W(e.  )+...+W(e-  ).   The  weight  of  G  is  defined  equal  to 
Jl         Jt 
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the  sum  of  the  weights  of  all  edges  in  G.   Let  T  =  {V  ,E 
FT,1\TT},  where  UrT  is  the  restriction  of  W  on  E™,  be  a  span- 
ning tree  of  G,  which  is  also  connected.   T  is  said  to  be 
a  Minimal  Spanning  Tree  (MST)  of  G  if  the  weight  of  T  is 
minimal  among  all  spanning  trees  of  G.   Figure  4.3  shows  an 
example  of  a  weighted  graph  G.   Figure  4.4  shows  the  corres- 
ponding MST  of  G. 

Definition  4.6. --Let  (V-^V^  be  a  partition  of  the 
vertex  set  V  of  a  weighted  graph  G  =  {V,E,F,W}.   The  weight 
W(-V1'V2^  across  the  partition  is  defined  as  the  smallest 
weight  among  all  edges  which  join  one  vertex  in  V   and  the 
other  in  V-j.   The  set  of  edges  ECV^V^  which  span  a  parti- 
tion will  be  referred  to  as  the  cut  set  of  {V  V?}  and  a 
link  is  any  edge  in  E(VlfV2)  whose  weight  is  equal  to  the 
weight  WCV-^V^.   The  set  of  all  links  in  ECV^V^  is  called 
link  set  LCV-^V^  of  {V^V,}. 

The  following  theorem  allows  us  to  find  an  MST  of  a 
weighted  graph  from  the  link  sets. 

Theorem  4. 2 . --An  MST contains  at  least  one  edge  from 
the  link  set  LfV^V^  of  every  partition  {V^V  }.  Every 
edge  of  an  MST  is  a  link  of  some  partition  of  V. 

Theorem  4.3  reveals  that  the  appropriate  clusters  can 
be  found  as  subtrees  of  any  MST. 

Theorem  4. 3. --If  Vj  is  a  non-empty  subset  of  V  with 

the  property  W(V.  ,V.  )  <  W(V  V-V.)  for  all  partitions 
12 
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Figure  4.3.   An  example  of  a  weighted  graph, 


Figure  4.4.  An  MST  of  the  weighted  graoh  shown  in  Figure 
4.3. 
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{V,  ,V.  }  of  V. ,  then  the  restriction  of  any  MST  to  V. 
ix   i2       1  x 

forms  a  subtree  of  the  MST. 

4.1.1.   Finding  an  MST  of  a  Weighted  Graph 

From  Theorem  4.2,  it  is  clear  that  an  MST  can  be  found 
from  a  connected  graph  by  building  up  a  subtree  T*,  to  which 
a  link  of  {VT,,VG-VT,}  is  added.   Let  m  be  the  total  number 
of  vertices  of  G.   We  can  set  VQ    =    {v  ,  ...,v  }.   Three  ar- 
rays are  required  to  achieve  the  purpose  of  finding  an  MST 
from  a  weighted  graph  G. L  J 

1.  Vertex  array  X:   It  indicates  which  vertices  are  in  V„ , , 
that  is,  if  X(i)  =  1,  then  v±    e   V  while  if  X(i)  =  0, 
then  v.  t   V„, . 

2.  Reference  array  R:  If  X(i)  =  1,  R(i)  specifies  the  in- 
dex of  the  vertex  v.  in  VT , ,  if  v,  is  adjacent  to  v.  in  T*. 
If  X(i)  =  0,  R(i)  specifies  the  index  of  vertex  v.  in  VT , , 

such  that  W(v,,v  )  =  min   {W(v.  ,vj  } ,  where  W(v. ,v  )  is  the 
x   J     vqeV-pt     1      q  i   q 

weight    of  the  edge  joining  vertices  v.  and  v  . 

1     q 

3.  Weight  array  Z:   Z(i)  is  equal  to  the  weight  of  the  edge 


incident  with  v.  and  v 


(i),  that  is,  Z(i)  =  WCvifvR(i)) 


The  link  of  {VT,,VG-VT,}  can  be  found  from  the  vertex 
array  X  and  the  weight  array  Z  by  noting  that  the  edge  con- 
necting a  vertex  v±   and  vR,.. ,  where  vi  is  not  in  the  sub- 
tree I"  of  MST  (i.e.,  X(i)  =  0),  has  a  weight  Z(i)  equal  to 

min     ^V(j)|X(j)  =  0}.   The  edge  is  then  added  to  T'  by 
j={l, . . . ,m} 
setting  X(R(i))  =  1.   Figure  4.5  shows  the  flow  diagram  for 

finding  an  MST  of  a  connected  weighted  graph. 
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Input 

a  weighted  graph 


Initialization : 
X(l)  =  1,  X(i)  =  0, 

i  =  2,...,m. 
R(i)  =  1  if  Vj  is  adjacent 

to  V!, 
=  0,  otherwise. 
Z(i)  =  W(vi,vRm)  if 

R(i)  f    6/ 
=  wn  if  R(i)  =  0, 
where  w^  >  all  weights  in 
the  graph. 
k  =  1. 


Find  j  such  that 

Z(j)  =  iuin{Z(i)  |X(i)  =  0}. 
Set  X(j)  =  1. 

If  X(i)  =  0  and  Z(i)  >  W(vi,Vj] 
set  R(i)  =  j  and  Z(i)  =  (vi,V-;) 
k  =  k+1. 


Figure  4.5.   Flow  diagram  for  finding  an  MST  of  a  weighted 
graph. 
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4.1.2.   Finding  Major  Paths 

As  is  easily  seen,  the  most  straightforward  clustering 
can  be  done  by  setting  a  threshold  w,  such  that  if  an  edge 
in  an  MST  is  of  weight  greater  than  w,  the  edge  is  deleted 
from  the  MST.   The  deletion  of  heavily  weighted  edges  from 
an  MST  will  yield  a  forest  of  subtrees.   Every  subtree  cor- 
responds to  a  cluster  of  the  connected  weighted  graph  G. 
It  should  be  pointed  out  that  an  MST  of  a  connected  weighted 
graph  is  not  unique.   From  Theorem  4.3,  it  is  known  that 
this  non-uniqueness  of  the  MSTs  does  not  restrict  the  forma- 
tion of  the  same  clusters  from  different  MSTs  of  a  graph. 
In  many  cases,  more  sophisticated  consideration  should  be 
taken  to  cluster  the  graph.   Statistics  of  the  weights  of 
edges  on  major  paths  should  be  taken  into  consideration. 
For  a  tree,  usually  there  exist  many  major  paths.   A  sys- 
tematic method  should  be  set  to  find  all  major  paths.   Well- 
distinguished  major  paths  are  of  interest.   Two  major  paths 
can  be  considered  as  well  distinguished  if  they  have  only  a 
small  portion  of  paths  in  common.   Also,  the  branches  from 
a  diametral  path  are  of  interest.   Since  a  tree  is  a  simple 
graph,  any  arc  in  a  tree  can  be  represented  by  a  sequence 
of  vertices.   The  following  are  useful  definitions. 

Definition  4. 7. --Let  T,  be  a  tree  of  diameter  I,    which 
is  even.   There  is  only  one  center  c  in  T, .   The  arc  a(v,c) 
from  a  leaf  v  to  the  center  c  is  called  a  radial  path.   If 
d(v,c)  =  1/2,    a(v,c)  is  called  a  maximal  radial  path.   Let 
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T2  be  a  tree  of  diameter  I,    which  is  odd.   There  are  two 
centers,  c,  and  c2,  in  T0.   For  any  leaf  v,  if  d(v,c)  > 
d(v,c.)>  where  i  f    j,  the  arc  from  v  to  c.,  a(v,c)>  is 
called  a  radial  path.   If  d(v,c.)  =  (£+l)/2,  a(v,c.)  is 
called  a  maximal  radial  path.   Let  a(v.,v.)  and  afv^.v.) 

r  ^1>^         ».  |.»  y 

be  two  arcs,  where  both  v.  and  v   are  leaves,  and  v.  be  the 
only  common  vertex  of  the  two  arcs.   If  d(v.,v.)  <_  d(v  ,v.)> 
then  a(v.,v.)  is  called  a  branch  and  any  arc  containing 
a(v  ,v.)  is  called  a  stem.   v.  is  called  a  branching  vertex. 

A  relation  R  is  defined  on  all  radial  paths  in  a  tree 
T.   Let  s,  and  s~  be  two  radial  paths  in  T.   If  s,  and  s, 
contain  a  common  subsequence  of  more  than  one  vertex,  then 
s,Rs0.   It  is  obvious  that  R  is  an  equivalence  relation. 

Let  T  be  a  tree  having  only  one  center  c  and  v.  ,...,v.   be 

Xl       Xt 
all  vertices  adjacent  to  c.   Every  radial  path  has  a  sub- 
sequence of  vertices  (v.  ,c)  for  some  vertex  v.  ,j=l,...,t. 
Hence  there  are  t  equivalence  classes  of  radial  paths  in- 
duced bv  the  equivalent  relation  R.   Let  S.  denote  all 

radial  paths  having  a  subsequence  of  vertices  (vj  ,  c).   Any 

j 
radial  path  of  S.  can  be  combined  with  any  maximal-radial 

path  of  S . ,  to  form  a  major  path,  where  j  f    j ' e{l ,  . . .  ,  t} . 

Let  s^J  =  fv^   ,  v^   ,...,v.   ,  c)  be  a  maximal -radial  path 
u     *•  u,   '  u?   '    '  i- ,  J  r 

in  S.,.   (v^,v^t...,v.  ,c,v.  ,...  ,v(],'\v(J'h    then 
3'    v-u1'u?'    '  i . '  '  i   »    *   u'2  *  U'j  J 

forms  a  major  path.   If  T  is  a  tree  having  two  centers,  c, 
and  c7,    every  radial  path  S  in  T  either  contains  the  sub- 
sequence of  vertices  (c,,c?),  or  contains  the  subsequence 
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of  vertices  (c^c-^.   Hence  there  are  two  equivalence  classes 

induced  bv  the  relation  R.   Let  S   =  fv   c . , c . 1  be  a 

u     ui       x   J 

radial  path  and  S  ,  =  (v  ,  ,...,v  ,        ,c,c)  be  a 

1      U  C*-l)/2   J   X 

maximal  radial  path.   Su  and  S  ,  can  be  combined  into  a 

major  path  (v   ,  .  .  .  ,  c  .  ,c . , v  ,        ,...,v  ,  ). 
1         3   U  CA-l)/2        1 
Because  of  less  storage  required  and  easy  combination 

into  major  paths,  the  storage  structure  of  radial  paths 

would  be  that  only  one  radial  path,  which  has  maximum  length 

among  all  radial  paths  in  the  same  equivalence  class,  of 

every  equivalence  class  is  stored  in  the  full  sequence. 

Any  other  radial  paths  are  stored  as  branches. 

Every  leaf  in  a  tree  initializes  a  sequence.   Trim  all 
the  leaves  from  the  tree.   If  the  adjacent  vertex  v.  of  a 
leaf  v.  does  not  turn  out  to  be  a  leaf  after  v.  is  trimmed, 
the  corresponding  sequence  will  represent  a  branch.   The 
sequence  having  v^   as  a  leaf  will  be  the  corresponding  stem. 
The  procedure  is  iterated  until  either  there  are  only  two 
vertices  left  or  there  is  only  one  vertex  left.   The  ver- 
tices finally  left  are  the  centers  of  the  tree.   Figure  4.6 
shows  the  flow  diagram  for  finding  radial  paths  and  centers 
in  a  tree. 

The  tree  shown  in  Figure  4.4  has  only  one  center  v.. 
The  diameter  of  the  tree  is  4.   The  set  of  radial  paths, 
having  full  sequences,  is  { (Vj , v3 , v4) , (v? ,v5 , v4) } .   06,v5) 
is  a  branch  depending  on  the  maximal  radial  path  (v7,v,,v.). 
02,v4)  is  a  branch  having  (v^v^v^  and  (v?,v r,v.)    as 
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Input 

a  nontrivial    tree 


For   every   leaf   in    the    tree, 
initialize    a   stack   which  has 
the    leaf   as    the    only   clement, 
Let   K  be    the    total    number   of 


leaves    in   the    tree. 


■ 

no 

Note  v^  be  the  top  element 

of  the  kth  stack. 

Note  Vj  adjacent  to  v^. 

Set  p(Vj)  =  p(Vj)-l, 

pOi)  =  p(Vi)-l. 

Put  Vj  in  the  kth  stack. 

Figure   4.6.       Flow   diagram   for   finding   radial   paths    and 
centers    in   a   tree. 


86 


Figure   4.6.       Continued 
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stems.   (v.,,v_,v.)  and  (v_,v5,v.)  can  be  combined  into  a 
diametral  path  (v,  ,  v.,  ,  v.  ,  v~  ,  v_)  .   From  the  branch  (v,,v<-) 
we  can  find  a  maximal  radial  path  (vc>vr>v,i)  which  is  de- 
pendent on  the  maximal  radial  path  (v.,\'  ,v.).   (v  v  v.) 
and  (v6»vc>v4)  can  be  combined  into  a  diametral  path 
(v,  ,  v_  ,  v.  ,v. ,  v,}  .   (v2'va)  ^-s  independent  o£  any  other 
radial  paths  in  the  tree,  hence  we  can  combine 

1.  (v2'v4^  with  ^vi'v3»v4^  t0  a  maJor  Path  Cv2,v4,v3,v1) , 

2.  (v2,v4)  with  (v7»v5>v-]  to  a  major  path  (v_ , v. , v5 , v_) , and 

3.  (v2>v4)  with  (v6,v5,v4)  to  a  major  path  (v2 ,v4 , v5 ,v6) . 


4.2.   The  Representation  of  a  Digitized 
Picture  by  a  Weighted  Graph 


A  digitized  picture  g  can  be  represented  by  a  weighted 
graph  G  in  the  following  manner.   Every  picture  point  is 
considered  as  a  vertex.   Several  possible  methods  are  used 
to  define  the  connection  of  vertices  and  the  weight  of  the 
corresponding  edges. 

1.  Method  1  is  that  every  picture  point  p  is  connected  to 
any  of  the  four-neighboring  picture  points  p'  with  a  weight 
l/(g(p)+g(p'))  except  that  g(p)+g(p')  =  0. 

2.  Method  2  is  that  every  picture  point  p  is  connected  to 
any  of  the  eight-neighboring  picture  points  p'  with  a  weight 
of  d(p,p')/(g(p)+g(p'))  except  that  g(p)+g(p')  =  0. 

3.  Method  3  is  that  every  picture  point  p  is  connected  to 
the  16 -neighboring  picture  points  p'  with  a  weight  of 


d(p,p*)/fg(p)+g(p'))  except  that  g(p)+g(pf)  =  0.   Figure 
4.7  shows  the  three  methods  for  connecting  nodes. 


method  1 


method  2 


method  3 


Figure  4.7.   Three  methods  for  connecting  picture  points 


Because  the  MST  will  he  invariant  under  the  monotone 
transformation  of  the  weights,  a  quantizer  can  be  used  to 
transfer  monotonically  the  weights  assigned  by  the  previous 
paragraph  to  a  subset  W  of  the  natural  numbers.   Let  g  be 
a  n-level  picture.   It  is  easily  seen  that  for  method  1, 
the  cardinal  number  of  W  is  2n-3;  for  method  2,  the  cardi- 
nal number  of  W  is  4n-5;  and  for  method  3,  the  cardinal 
number  of  W  is  6n-7. 

The  weighted  graph  G,  representing  a  multi-level 
picture,  is  not  necessarily  a  connected  graph.   Once  the 
weighted  graph  G  of  a  digitized  picture  is  found,  the  flow 
diagram  shown  in  Figure  4.1  can  be  applied  to  G  to  find  an 
MST.   The  edges  in  the  MST  found,  having  weights  equal  to 


89 


the  initial  weight  w    are  deleted.   Hence  the  result  would 
be  a  minimal  spanning  forest. 

For  the  same  reason  of  encoding  a  curve  by  the  chain 
code  (discussed  in  detail  in  Chapter  II) ,  chain  codes  are 
used  to  represent  edges,  instead  of  using  a  pair  of  vertices 
or  assigning  a  name  to  an  edge.   In  method  1,  from  a  vertex 
to  another  vertex,  there  are  only  four  possible  edges, 
hence  a  2-bit  chain  code  is  used  to  encode  edges.   In  method 
2,  from  a  vertex  to  another  vertex,  there  are  only  eight 
possible  edges,  hence  an  octal  chain  code  is  used  to  encode 
the  edges.   In  method  3,  there  are  only  16  possible  edges 
from  a  vertex  to  another  vertex,  hence  a  hexadecimal  chain 
code  is  used  to  encode  the  edges. 


CHAPTER  V 
FEATURE  EXTRACTION 

As  stated  in  Chapter  I,  feature  extraction  strongly 
depends  on  the  type  of  pictures  handled.   Here  in  this 
chapter,  we  are  discussing  only  area  pictures.   There  are 
mainly  two  types  of  features:   local  features  and  global 
features.   Local  features  are  the  ones  which  are  dependent 
on  the  individual  objects  and  are  independent  of  any  other 
objects,  for  example,  area,  centroid,  shape  of  object, 
principal  axis  direction,  elongation  index,  etc.   Global 
features  are  the  ones  which  describe  the  interrelationship 
of  objects  in  the  picture,  for  example,  the  inclusion  rela- 
tion of  objects  in  the  picture,  overlapping,  touching  of 
objects  in  the  picture,  the  distribution  of  objects  in  the 
picture,  etc.   Assume  that  the  objects  are  described  by 
the  boundary  contours.   The  boundary  contours  are  described 
by  the  octal  chain  codes.   An  object  which  has  one  exterior 
contour  CQ  and  q  interior  contours  C,  ,...,C   can  be  de- 

q  q 

scribed  by  Rr   -   U   R    where  Rr   is  the  region  enclosed 

L0    j=l   Lj         Cj 
by  the  contour  C,  j  =  0,...,q,  where  -  is  the  substraction 

in  the  set  theory  and  U  is  the  union. 
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5.1.   Some  Fundamental  Local  Features 

The  most  important  and  useful  local  features  are  areas 
centroids,  shapes,  major  principal  axis  directions  and  the 
corresponding  elongation  index  of  objects.   We  are  attack- 
ing the  problems  one  by  one.   Remember  that  the  boundary- 
contours  are  described  by  the  start  points  and  the  associ- 
ated sequences  of  octal  chain  codes 


5.1.1.   Area 

Since  the  contours  are  encoded  into  a  sequence  of 

straight  lines,  the  area  inside  the  contour  is  the  algebraic 

sum  of  areas  of  the  strips  between  the  line  segments  and  the 

x  axis.   Figure  5.1  shows  the  x-y  coordinate  system  and  the 

eight  possible  line  segments  with  the  corresponding  octal 
chain  codes. 


• 

,   / 

1 
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->■          * 
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Figure  5.1.   X-y  coordinate  system  and  the  eight  possible 
line  segments  with  the  corresponding  octal 
chain  codes. 
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The  following  is  a  table  of  the  area  a.  of  the  strip 
r.  between  a  segment,  corresponding  to  the  octal  chain  code 
si  which  starts  at  (x^y^,  and  the  x  axis,  and  the  incre- 
ment Ax_L  and  Ayi  in  the  x  and  y  coordinates,  respectively. 
From  Table  S.l  we  can  get  a.±    =  Axi  (yi  +  Ay./2).   The  sign 
of  the  area  enclosed  by  a  contour  is  positive  if  the  contour 
is  encoded  in  the  clockwise  direction  and  is  negative  other- 
wise.  The  area  of  an  object  which  is  represented  by 

q  q 

C   "  ._   RC.  1S  then  eq.ual  to  iarea(.Rc  )|  -  I      |area(R   )|. 
The  ratio  |area(R   )j/|area(Rr  ) |  is  a  normalized  measure- 
ment  of  the  size  of  the  hole  enclosed  by  the  interior  con- 
tour C.  ,    j=l,.. . ,q. 

Table  5.1 
Table  of  area,  x-increment  and  y-increment 

Octal  chain  code      Area     x-increment   y-increment 
Si ai  A*i  Ayi 


0 
1 
2  0  0  1 


■y±  -i  o 

(y^l/2)        -i  i 


3  '  Y±+l/2  1  i 

4  110 

5  7-1/2  1  -i 


6 


0  -1 


7  -C/i-1/2)         -1  -1 
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5.1.2.   Centroid 

The  centroid  of  a  region  enclosed  by  a  contour  C  can 

be  found  by  the  mean  value  technique.   The  y  coordinate  of 

m 

the  centroid  is  v  =  (  I   a.  y-)/A,  where  a.  is  the  area  of 

-i   l  J  iJ  l 

i=  1 

the  strip  between  the  corresponding  segment  and  the  x  axis, 

y.  is  the  mean  y  coordinate  of  the  strip,  m  is  the  length 

m 


of  the  sequence  representing  C  and  A  =   I   a-  is  the  area 

1-1 

of  the  region.   Table  5.2  gives  the  a.  y.  values  for  the 

&  °  i  J  i 


eight  possible  segments.   A  formula  can  be  derived,  a-  y. 
=  Ax^y?  +  Ay^'i  +  Ayi/3))/2. 


Table  5.2 
Table  of  moment 


Octal  chain  Moment  of  a^           Moment  of  a^t 

code  about  the  x-axis  about  the  y-axis 

s  -  a  -  y  •                a !  x ! 
1 1  ;  1 1   1 

0  -v?/2  0 

1  -(7-    +   Yi    +   l/3)/2  Cx?    -   x±   *   l/3)/2 

2  0  x?/2 

3  (y-    +   Y±   +    l/3)/2  [x?   +   xi    +   l/3)/2 


4  y^/2  0 

5  (y-  -  yi  +  l/3)/2  -(x?  +  x±   +  l/3)/2 

6  0  -x?/2 

7  -Cy-  -  Yi    +  l/3)/2  -(x?  -  xi    +  l/3)/2 
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m 


wh 


The  x  coordinate  of  the  centroid  is  x  =   E   a-  x!/A' 
ere  a!  is  the  area  of  the  strip  between  the  corresponding 


segment  and  the  y  axis,  x.'  is  the  mean  x  coordinate  of  the 
m        1 


strio  and  A'  =   E   a.'  is  the  area  of  the  region  enclosed  bv 
the  contour  C  which  has  opposite  sign  from  that  of  A.   By- 
symmetry  a  formula  can  be  derived, 

a!  x!  =  Ay.fx?  +  Ax. (x.  +  Ax./3))/2. 
11     '  l v  l      i^i      l '  •*  J  ' 

For  an  object  which  has  holes,  the  net  centroid  is 


x  =  C|AC  |xc   -   E  |A   |x   )/A 
0    0    j  =  l    j    j 

and     y  =  (|AC  | yQ   -   E   | A   | y   )/A, 
0   0   j=l    j    j 

where  A„   is  the  area,  (x„  ,  y_.  )    is  the  centroid  of  the 

j  j     j 

region  enclosed  by  the  contour  C.,  j=0,l,...,q  and  A  is  the 

net  area  of  the  objects. 


5.1.3.   Shape 

The  shape  of  an  object  can  be  described  by  the  shape 
of  the  exterior  boundary  contour  of  the  object  and  the  shapes 
of  interior  boundary  contours  of  the  object  if  they  exist. 
A  method  to  describe  the  shape  of  a  contour  is  then  required. 
Let  r.s  consider  a  contour  in  the  real  plane  first.   Second 
order  differentiation  can  very  well  describe  the  shape  of 
the  contour  because  it  can  partition  the  contour  into  con- 
cave and  convex  portions.   In  the  digitized  picture,  the 
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octal  chain  code  describes  the  first  order  differentiation 

of  the  curve.   The  difference  c  '-    c'  of  two  successive  octal 

chain  codes  c  and  c1,  which  is  defined  in  Section  3.2,  i.e., 

c1c'  =c-c'+8p,  where  p  is  an  integer  such  that  the 

absolute  value  of  the  difference  is  less  than  or  equal  to  4, 

can  describe  the  second  order  differentiation. 

Let  Cq  ,c,  , . . . , c   ,  be  the  sequence  of  octal  chain  codes 

representing  the  contour  C  and  X  =  x„,...,x  -,  ,    where  x-  = 
r        s  0 '    '  m-1'         i 

c-.,  -  c  ,  i  =  0,  l,...,m-l  and  c   =  cn  be  the  correspond- 
l+l    i '       '   '    '         m    U  r 

ing  difference  sequence.   Because  of  the  quantization  error 

and  noise  in  the  picture,  high  frequency  noise  appears  in 

f  321 
the  X  sequence.   A  digital  filter^   '  is  then  required  to 

filter  out  this  high  frequency  noise.   A  Manning  window 

function  W,  which  is  defined  as 

wk  =  1  -  cos(2^k/K)  ,  0  <  k  <  K, 

is  used  here  to  convolve  the  input  sequence  X  to  filter  out 
the  high  frequency  noise.   The  output  Y  of  the  digital 
filter  W  is  then 

K 
^   '  ^mod  m   j  =  0   J   *■   J-^mod  m 

i  =  0,...,m-l.   The  output  sequence  Y  is  called  the  smoothed 
difference  sequence. 

The  change  of  signs  between  successive  y- 's  in  the  Y 
sequence  is  important  to  note.   If  the  value  of  y.  is  posi- 
tive, the  i  *  point  in  the  contour  is  in  the  concave  portion 
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of  the  contour.   If  the  value  of  y.  is  negative,  the  i 
point  in  the  contour  is  in  the  convex  portion  of  the  contour. 
The  Y  sequence  is   periodic   because  the  last  element  in  Y 
is  one  ahead  of  the  first  element  in  Y.   We  can  thus  group 
the  Y  sequence  into  a  sequence  of  t  subsequences  Y„,...,Y   , 
such  that  Y  =  Y„  ...  Y. _, ,   The  signs  of  all  y.  values  in  a 
subsequence  are  the  same  and  the  signs  of  y.  values  in  a 
subsequence  are  different  from  those  of  y.  values  in  an 
adjacent  subsequence. 

Let  N  =  (nn,...,n   -, )  be  the  sequence  such  that  the 
n.    point  in  the  contour  is  the  start  point  of  the  Y.  por- 
tion of  the  contour,  i  =  0,...,t-l.   We  call  these  points 
the  critical  points.   It  is  noted  that 

Yi  "  ()'ni'1'(V1Vod  m-""yCn(i  +  i;|      1}     ), 

v     mod  e    mod  m 

i  =  0,... ,t-l. 

A  sign  specification  a  should  be  accompanied  with  the 
N  sequence  such  that  a  =  sgn(y   ).   From  the  value  of  a  we 

no  th 

can  tell  that  the  portion  of  the  contour  from  the  n.    point 

to  the  n^.  ...        point,  i  =  0,...,t-l,  is  a  concave  por- 
(i+l)        ft  >    >    >  f 

v    •'mod  t 
tion  or  a  convex  portion.   If  a  =  +1  and  if  i  is  even  (odd), 

it  implies  that  sgn(y   )  =  +1  (-1).   Hence  the  portion  of 

the  contour  from  the  n.    point  to  the  n,-  ...        point 

v    J mod  t 
is  a  concave  (convex)  one.   If  a  =  -1,  the  situation  is 

just  the  opposite  to  that  of  a  =  +1. 
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The  window  size  K  of  the  Harming  filter  is  chosen  such 

that  if  K1  ^  K  is  used  as  the  window  size,  then  for  every 

contour  C  in  the  picture,  any  two  successive  critical  points 

are  within  6  length  along  the  contour  5,  where  4  <  6  <  9  is 

a  threshold,  and  if  K'  <  K  is  used  as  the  window  size,  then 

there  exists  a  contour  C  having  the  N  sequence  (if  t  f    1) 

such  that  (n.  -  n.  ,)   ,    <  5  for  some  1  <  i  <  t,  where  m 
i     l-l  mod  m  —  —   —   ' 

is  the  length  of  the  contour  C. 

Let  the  i    contour  point  have  a  local  maximum  |y. | 
value  and  |y- |>a.   The  i    contour  point  is  said  to  be  an 
extreme  point.   If  y.  >  0,  then  the  i    contour  point  is  on 
a  concave  portion.   Hence  the  i    contour  point  is  also 
called  a  valley  point.   If  y.  <  0,  the  i    contour  point 
is  also  called  a  peak  point. 

If  there  is  one  or  less  extreme  point  between  two  sue- 

*t*  V*  +■  Vi 

cessive  critical  points,  the  n.  -,    and  the  n.    points, 

the  curvature  of  the  portion  of  the  contour  between  the 

successive  critical  points  can  well  be  defined  as 

sgnfy    )((n.  -  n-  ,]   ,   /d- fn.  ,  n.  , ) -1)  ,  where 
5  wn.  -,     l     i-l^mod  m   r  i  '   i-ly   '' 

i-l  th 

d(n.  -  n.  -, )  is  the  Euclidean  distance  between  n.  -,    and 
*•  l    ±-lJ  i-l 

the  n-    points. 

It  should  be  pointed  out  that  the  Euclidean  distance 

between  the  starting  point  and  the  end  point  of  a  curve  can 

be  found  without  the  specification  of  the  locations  of  the 

starting  point  and  the  end  point,  if  the  octal  chain  code 

sequence  of  the  curve  is  known.   Let  c,  ...  c  ,  be  the 
n  1      m 
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octal  chain  code  sequence  of  a  curve  C.   We  can  calculate 

the  x(y) -projection  of  the  vector  from  the  start  point  of 

C  to  the  end  point  of  C  by  summing  up  the  x(y)  increments 

contributed  by  the  octal  chain  codes  as  defined  in  Table 

m'  m' 

5.1,  i.e.,  AX  =   E   Ax-,  AY  =   E   Ay..   The  Euclidean  dis- 

i=l    x       i=l    x 
tance  between  the  start  point  and  the  end  point  of  the  curve 

-      7  m '      -      m'      ~ 

C  is  then   AXZ  +  AYZ  =   (  S   Ax.) l    +    (  E   Ay . )  . 

i=l   x      i=l   x 

5.1.4.   Principal  Axis  Direction  and 
Elongation  Index 

If  a  contour  C  has  no  very  deep  concavity,  we  can  as- 
sume that  the  region  enclosed  is  an  elliptical  one.   Gener- 
ally there  are  two  principal  axes  of  the  region  through  the 
centroid.   In  order  to  find  the  principal  axes,  we  have  to 

find  1,1,  the  moments  of  inertia  of  the  region  about  the 
x'   y '  b 

x  and  y  axes,  respectively,  and  P   ,  the  product  of  inertia 
of  the  region.   Here  the  x-y  coordinate  system  is  the  trans- 
lation   of  the  old  x-y  coordinate  system  obtained  by  set- 
ting the  centroid  of  the  region  as  the  new  origin.   Note 

that  both  I   and  I   are  positive  and  are  given  by 
n     — j-       Y  n     — j  — j 

I   =   E   a-  v.   and  I   =   E   a!  x!  ,  where  a-  y.   is  the 

i  =  l  7    i  =  l  — j 

moment  of  inertia  of  a-  about  the  y-axis  and  a!  x!   is  the 

moment  of  inertia  of  a.  about  the  x-axis.   P    =  sgn(A) 

l  xy     b    ^  J 

n     ' 

E   a-  x.y.,  where  a.  x.y.  is  the  product  of  inertia  of  a-. 
.  ,   l   l'l*        ii^i        v  l 

1  =  1  T         o- 


Table  5.3  shows  the  a-  v.  ,  a!  x!   and  a-  x.y.  values 

3-1'   11         1171 

Formulas  for  those  calculations  can  be  derived: 
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aiyi2  ■  Axi(4yi3  +  Ayi(6yi2  +  UAy-^y.  +  1))/12, 
a!x! 2  =  Ay. (4x.3  +  AX-C6X.   +  4(Axi)xi  +  1))/12,  and 

aiXiyi  =  Axi(Xi  (2x  4  Axi))/4  +  (AxiAyiJXiyi/2 
^j(Ayi)yi/3  ^AxJx^G  +  1/8. 

Let  9  be  the  angle  o£  rotation  from  the  x,y  axes  to 
the  two  principal  axes  x'  and  y',  respectively,  as  shown 

in  Figure  5.2. 

f  33") 
A  formula1  J    has  been  derived  such  that  tan  2  8  = 

-2P   /{I  -I  ")  .   Hence  we  can  find  this  angle  of  rotation 
xy  v  y  x-  6 

very  easily.   A  major  principal  axis  is  defined  as  the  axis 
about  which  the  moment  of  inertia  is  minimum.   Hence,  in 
order  to  find  the  major  principal  axis,  we  have  to  calcu- 
late I., and  I  ,,and  to  find  out  which  has  less  value.   The 

formula  for  I  ,  and  I  ,  are: 
x       y 

Ix,  =  Clx+Iy)/2  +  ((Ix-ly)/2)*  cos20  -  Pxy  sin26 

and      I  ,  =  (Ix+Iy)/2  -  C(Ix-Iy)/2)*  cos29  +  ?xy   sin  29. 

The  major  principal  direction  can  be  represented  by  8  or 
0+^/2  depending  on  x'  or  y'  being  the  major  principal  axis, 
respectively.   The  ratio  of  the  moment  of  inertia  about  the 
major  principal  axis  to  the  moment  of  inertia  about  the 
minor  principal  axis  is  called  the  elongation  index.   The 
elongation  index  ranges  from  0  to  i.   It  indicates  how 
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(x,y) 


Figure  5.2.   Principal  axis  direction, 
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sharp  the  region  is.   The  less  the  elongation  index,  the 
sharper  the  region  is.   For  example,  if  the  elongation  index 
is  1,  the  region  is  a  circle,  x^hile  if  the  elongation  index 
is  almost  0,  the  region  is  almost  approaching  a  line. 

5.2.   Global  Features 

The  global  features  are  the  ones  which  describe  the 
interrelationship  of  objects  in  the  picture.   Since  some- 
times an  object  may  contain  several  holes,  we  can  consider 
the  regions  enclosed  by  the  boundary  contours  as  individual 
objects  and  find  the  global  features  among  the  regions  to 
describe  the  object  itself.   To  find  the  global  features 
among  objects,  only  the  regions  enclosed  by  the  exterior 
boundary  contours  need  to  be  considered.   The  following  are 
several  nontrivial  global  features. 

5.2.1.   Inclusion  Relation  Among  Objects 

Inclusion  relations  among  objects  can  be  derived 
directly  from  the  inclusion  relation  among  contours.   Let 
H  be  the  Hasse  graph  representing  the  inclusion  relation 
among  contours  and  N'  be  the  set  of  all  nodes  in  H  which 
correspond  to  exterior  contours  of  objects  in  the  picture. 
Let  H'  be  the  graph  which  represents  the  transitive  reduc- 
tion of  the  reachability  among  nodes  in  N'  revealed  by  the 
graph  H.   H"  is  then  a  Hasse  graph  which  represents  the  in- 
clusion relation  among  objects.   Since  there  is  a  one-to-one 
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correspondence  between  exterior  contours  of  objects  and 
objects,  the  nodes  in  N'  are  in  one-to-one  correspondence 
with  the  objects. 

5.2.2.   Distribution  of  Objects  in  the  Picture 
(With  Consideration  to ~tne  Distances 
Between  Objects)- 

The  distance  between  two  objects  0  and  0'  is  defined 
as  the  Euclidean  distance  between  the  corresponding  cen- 
troids  o  and  o'.   One  method  to  describe  the  distribution 
of  objects  in  the  picture  is  using  the  Minimal  Spanning 
Tree  (MST)  technique,  which  is  discussed  in  detail  in  Chap- 
ter IV.   A  K-nearest-neighbor  graph  G  =  (N,  E,  W)  can  be 
set  by  the  following  rules: 

1.  For  any  object  0  in  the  picture,  there  exists  one  and 
only  one  node  n  in  N. 

2.  There  exists  an  edge  connecting  nodes  n  and  n'  if  either 
0'  is  a  K-nearest-neighbor  of  object  0  or  0  is  a  K-nearest- 
neighbor  of  object  0',  where  objects  0  and  0'  are  corres- 
ponding to  nodes  n  and  n' ,  respectively. 

3.  Let  e  be  an  edge  connecting  nodes  n  and  n',  the  weight 
W(e)  assigned  to  edge  e  is  defined  as  the  distance  between 
objects  0  and  0',  where  0  and  0'  are  the  objects  corres- 
ponding to  nodes  n  and  n1 ,  respectively. 

Let  T  be  an  MST  of  the  weighted  graph  G.   T  can  be 
found  by  the  method  presented  by  the  flow  diagram  4.1.   T 
can  reveal  the  information  describing  gestalt  clusters  of 
objects  in  the  picture.   The  philosophy  is  the  same  as 
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stated  in  Chapter  IV,  except  we  consider  every  object  in 
the  picture  as  an  individual  vertex. 

The  definition  of  the  distance  between  objects  stated 
in  the  last  paragraph  is  a  good  one  if  the  object  size  and 
shape  are  quite  uniform  through  the  picture.   If  the  objects' 
sizes  and  shapes  are  not  uniform  through  the  picture,  the 
distance  between  two  objects  0  and  0'  may  better  be  defined 
as  min{dCp,p') [peO  and  p'eO1}. 

5.2.3.   Distribution  of  Objects  in  the  Picture 

(With  Consideration  to  Both  the  Distances 
Between  Objects  and  Relative  Principal 
Axes'  Directions  .Among  Objects) 

If  all  objects  in  the  picture  are  quite  elliptical  in 
shape,  to  our  visual  systems  the  reasonable  gestalt  clusters 
of  objects  should  take  not  only  the  distance  between  objects 
into  consideration,  but  also  the  angle  between  the  principal 
axes  of  objects  into  consideration.  One   practical  example 
is  the  cluster  of  cells  in  the  epidermis,  which  will  be 
discussed  in  detail  in  Chapter  VI. 

The  method  used  to  describe  the  distribution  of  objects 
by  considering  not  only  the  distance  between  objects,  but 
also  the  relative  principal  axes'  directions  among  objects 
is  the  same  as  the  one  presented  in  the  last  section,  except 
in  setting  the  K-nearest-neighbor  weighted  graph,  the  weights 
assignment  should  be  modified  (i.e.,  rule  3  should  be 
changed).   Let  e  be  an  edge  connecting  nodes  n  and  n',  the 
weight  W(e)  assigned  to  edge  e  is  defined  as  d(0,0').  + 
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Y9(0,0'),  where  d(0,0')  is  the  distance  between  objects  0 
and  0',  3(0,0')  is  the  angle  between  the  major  principal 
axes  of  objects  0  and  0'  and  0  and  0'  are  the  objects  cor- 
responding to  nodes  n  and  n',  respectively. 

The  weighted  graph  in  the  last  section  is  a  special 
case  of  the  weighted  graph  in  this  section  under  the  assump- 
tion that  y  =  0. 


CHAPTER  VI 
EXPERIMENTS  AND  CONCLUSIONS 

All  techniques  stated  in  the  previous  chapters  have 
been  implemented  by  FORTRAN  programs.   Biomedical  images 
are  chosen  as  the  experiment  data.   It  should  be  pointed 
out  that  the  techniques  are  not  restricted  to  biomedical 
images.   The  reason  that  the  author  chose  this  special  type 
of  pictures  is  due  to  the  great  need  for  the  handling  of 
biomedical  images.   Chromosome  pictures,  skin  cell  pictures 
and  blood  cell  pictures  are  the  main  pictures  we  work  with 
in  the  experiments. 

6.1.   Experiments  With  Chromosome  Pictures 

Figure  6.1  is  an  8-level  digitized  chromosome  pic- 

(2  3^ 
ture.    J      Figure  6.2  is  the  enhanced  picture  and  Figure  6.3 

is  the  boundary  picture  obtained  by  the  programs  based  on 
the  gradient  technique.   There  is  only  one  boundary  in  the 
picture  with  a  length  equal  to  207.   The  corresponding  se- 
quence of  octal  chain  codes  is 

4443312011010101010  0011010121210121345454555545  5  4545 
445454456544332111010  0000101110  001012110112223  32  34  33 
4  334  33321000  70  7  7  70  77  76  70  700132  3332  33334  333  32100  70  7  77 
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Figure  6.1.   An  8 -level  picture  of  a  human  chromosome 
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Figure    6.2.      The    enhanced  picture    of   Figure    6.1. 
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Figure    6.3.      The   boundary  picture    obtained    from   Figure    6.2 
by'the    gradient   method. 
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077  76  7  75  766  7666665  7566  756556656556544554445  545545555. 
The  optimal  Hanning  window  filter,  used  to  filter  out 
the  noise  in  the  sequence  of  differences  of  successive  octal 
chain  codes,  has  a  window  size  equal  to  26.   The  filtered 
sequence  of  differences  of  successive  octal  chain  codes  in 
shown  in  Figure  6.4.   There  are  six  critical  points,  the 
17th,  45th,  77th,  104th,  122nd  and  140th  points  on  the  con- 
tour.  The  5th,  65th,  113th  and  148th  points  on  the  contour 
are  the  peak  points.   The  33rd,  97th  and  131st  points  in 
the  contour  are  the  valley  points.   The   parameter  is  +1, 
which  implies  that  the  portions  (17,  44),  (77,  103)  and 
(122,139)  are  the  concave  portions  and  the  portions  (45,  76), 
(104,  121)  and  (140,  16)  are  the  convex  portions.   The  por- 
tion (i,  j)  denotes  the  portion  of  the  contour  from  the  ith 
point  to  the  j    point  in  the  contour. 

Figure  6.5  is  the  labeled  output  picture  of  Figure  6.1 
using  the  contour  analysis.   Figure  6.6  is  the  contour  pic- 
ture obtained  by  the  method  stated  in  Chapter  III.   There 
is  only  one  object  contour  in  the  final  picture  with  a 
length  equal  to  201.   The  corresponding  sequence  of  octal 
chain  codes  is 

44444  32  2111001101010010110112112  345555  45545454544454 
55  564  34322  21101000010010110011011111122  53333434  334  33 
2210000  770  7  7  70  76  77012  32  3332  34  3334  322100  70  70  770  76  776  7 
6666  766  56  766  566  7565  5665665  5  545  54  445545  45  5  5456. 
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Figure    6.5.      The    labeled  picture    obtained    from    Figure    6.1. 
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Figure    6.6.      The   boundary   picture    obtained   from   Figure    6.1 
by. the    contour    analysis. 
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The  optimal  Harming  window  filter,  used  to  filter  out 
the  noise  in  the  sequence  of  differences  of  successive  octal 
chain  codes,  has  a  window  size  equal  to  28.   The  filtered 
sequence  of  differences  of  the  successive  octal  chain  codes 

is  shown  in  Figure  6.7.   There  are  six  critical  points,  the 

nr.th   rfrth   -74th   noth   ,,,th    .  ,__rd  ., 

19   ,45   ,74   ,98   ,  116    and  133    points  in  the  con- 
tour.  The  6   ,  62   ,  107    and  141    points  in  the  contour 
are  the  peak  points.   The  31   ,  90    and  125    points  in 
the  contour  are  the  valley  points.   The  a   parameter  is  +1, 
which  implies  that  the  portions  (19,  44),  (74,  97)  and  (116, 
132)  are  the  concave  portions  and  the  portions  (45,  73), 
(98,  115)  and  (133,  18)  are  the  convex  portions.   It  should 
be  pointed  out  that  the  dissimilarity  of  the  contours  ob- 
tained by  the  gradient  method  and  the  contour  analysis  is 
equal  to  0  as  is  expected. 

Method  2  as  stated  in  Section  4.2  is  used  to  find  the 
weighted  graph  of  Figure  6.1.  An  MST  has  been  found  as  shown 
in  Figure  6.8.   The  radius  of  the  MST  is  rn  =  51.   There  are 
two  centers  (23,  7)  and  (24,  7)  in  the  MST.   a  is  set  as 
1/4,  hence  any  branch  of  length  less  than  [otrn]  =  12  is  re- 
moved.  Only  four  radial  paths  (including  two  maximal  radial 
paths)  are  remaining,  which  are  shown  in  Figure  6.9. 

Radial  path  1: 

start  point  =  (6,  34), 

chain  code  sequence  =  00066006002020202020020020200 

00200202020220222222, 
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Figure  6.8.  An  MST  obtained  from  Figure  6.1 
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Figure  6;9.   The  four  radial  paths  in  Figure  6.8  obtained 
by  deleting  small  branches. 
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length  =  51  (i.e.,  it  is  a  maximal  radial  path). 

Radial  path  2: 

start  point  =  (15,39), 

chain  code  sequence  =  7006660002020200020002002002 

00202002200202000  200066, 

length  =  51  (i.e.,  it  is  a  maximal  radial  path). 

Radial  path  3: 

start  point  =  (46,25), 

chain  code  sequence  =  7766606006060060060606060660 

66666600066, 

length  =  39. 

Radial  path  4: 

start  point  ■  (50,  20)  , 

chain  code  sequence  =  7006606006060060660606600660 

6666666666666066022, 

length  =  47. 

Radial  path  3  is  dependent  on  maximal  radial  path  2,  while 
radial  path  4  is  dependent  on  maximal  radial  path  1.   (22, 
7)  and  (25,  10)  are  the  only  two  branching  vertices  in  the 
MST  with  the  removal  of  small  branches.   The  octal  chain 
code  sequence  of  the  path  from  (22,  7)  to  (25,  10)  is  222444 
which  is  of  length  6.   Since  this  length  is  much  smaller 
than  the  length  of  any  radial  path,  the  skeleton  of  the 
picture  can  be  considered  as  consisting  of  four  arms.   Each 
arm  is  a  subpath  of  a  relative  diametral  path  from  a  leaf 
to  the  nearest  branching  vertex.   The  branching  vertices 
are  shown  by  e> ,  and  the  centers  are  shown  by  •  in  Figure  6.9. 
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6.2.   Experiments  on  Skin  Cell  Pictures 

It  is  known ^  s    that  there  are  two  types  of  cells  in 
the  epidermis,  basal  cells  and  squamous  cells.   If  the  epi- 
dermis is  normal,  the  basal  cells'  nuclei  are  generally 
perpendicular  to  the  dermis-epidermal  junction,  and  the 
squamous  cells'  nuclei  are  parallel  to  the  epidermal-dermal 
junction.   The  basal  cell  carcinoma  will  appear  in  an  island 
The  main  features  of  the  tumor  cells  in  the  island  are: 

1.  the  nuclei  are  crowded,  and 

2.  there  is  no  uniform  pattern  of  the  alteration  of  the 
major  principal  axis  direction  of  the  nuclei. 

The  hair  follicle  shaft  can  be  distinguished  from  the 
basal  cell  carcinoma  island  because  the  hair  follicle  has 
an  acellular  protein  part  which  makes  the  hair  follicle  look 
like  a  doughnut  instead  of  a  solid  island.   The  sebaceous 
glands  can  be  easily  distinguished  from  the  basal  cells 
carcinoma  in  that  the  sabaceous  cells'  nuclei  are  quite 
widely  separated.   All  other  structures  in  the  skin  are 
characterized  by  having  scattered  nuclei  or  being  in  the 
form  of  a  small  island. 

In  our  experiment  we  first  extract  nuclei  from  the  pic- 
ture.  Figure  6.10  is  a  portion  of  the  skin  cell  picture 
obtained  through  the  quantization  of  the  PIDAC.   Contour 
analysis  is  used  to  extract  objects  from  the  picture.   Fig- 
ure 6.11  shows  the  intermediate  result,  which  is  the 
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Figure  6.10.   An  8-level  picture  of 
cell  picture. 
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Figure    6.11.      The    labeled  picture    obtained   from    Figure    6.10 
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labeled  picture  of  Figure  6.10.   Since  this  is  only  a  por- 
tion of  the  epidermis,  the   cytoplasma  part  touches  the 
picture  frame,  which  is  then  not  under  consideration.   It 
should  also  be  pointed  out  that  the  cytoplasma  parts  of  the 
cells  are  merged  together,  hence  it  is  very  hard  to  tell 
the  cytoplasma  boundary  of  each  individual  cell.   The  ob- 
jects extracted  are  then  only  the  nuclei  as  is  expected. 
Figure  6.12  shows  (nuclei)  object  boundaries.   Picture 
points  which  have  the  value  i  =  1,...,9  are  the  boundary 
points  of  object  i.   There  are  in  total  9  nuclei  in  the 
picture . 

Local  features  of  individual  nuclei  are  then  found  by 
the  methods  stated  in  Chapter  V.   Table  6.1  shows  the  re- 
sults of  the  local  features  of  the  9  nuclei.   XBAR  and  YBAR 
specify  the  location  of  the  centroid  of  an  object.   AREA  is 
the  area  of  an  object.   THETA  is  the  angle  (in  radians)  of 
the  major  principal  axis  of  an  object  with  respect  to  the 
positive  y  axis.   The  elongation  index  of  an  object  is  what 
is  defined  in  Section  5.1.4. 

The  distribution  of  nuclei  in  the  picture  is  described 
by  the  MST  as  stated  in  Section  5.2.3.   The  weight  assigned 
to  the  edge  connecting  two  nodes  n  and  n'  is  D(0,0')  + 
Y8(O,0'),  where  0  and  0'  are  the  nuclei  corresponding  to  the 
nodes  n  and  n',  respectively.   The  appropriate  MST  (i.e., 
the  appropriate  y    value)  is  found  through  the  flow  diagram, 
shown  in  Figure  6.13,  by  changing  y   value  until  proper 
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Figure  6.12.   The  boundary  picture  obtained  from  Figure 
6.10  by  the  contour  analysis. 
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Table  6.1 
Local  features  of  nuclei  in  Figure  6.12 


Nucleus 

XBar 

YBar 

Area 

Theta 

Eccentricity 

1 

11.6728 

21.5994 

54.5 

0.18481 

0.455348 

2 

22.1863 

7.7279 

68.0 

1.03434 

0.407566 

3 

35.4360 

22.8333 

47.0 

-0.91540 

0.325312 

4 

22.7193 

23.9228 

47.5 

-1.02386 

0.553201 

5 

47.1780 

9.0922 

79.5 

-0.06773 

0.377898 

6 

53.7212 

18.4975 

33.5 

0.06664 

0.212766 

7 

61.1953 

5.9293 

49.5 

-0.07382 

0.145627 

8 

62.8950 

25.9561 

38.0 

1.15331 

0.921100 

9 

66.3389 

14.0645 

31.0 

1.15437 

0.869634 

125 


Input 

centroids,  major  principal  axes 

directions  of  nuclei 


y   =  0 

ay  =  10 


Find  the  weighted  graph 
with  D(0,0*)+y6(0,0') 
assigned  as  weight  to 
the  edge  connecting  tA^o 
nodes  corresponding  to 
the  two  nuclei  0  and  0* 


Y+Ay 


Figure  6.13.   Flow  diagram  of  finding  the  distribution  of 
objects  in  Figure  6.10. 
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clusters  are  obtained.   The  clustering  method  used  in  Figure 

6.13  is  that  if  W(e)  >  w+aa  ,  then  the  edge  e  is  erased, 

where  w  is  the  mean  weight  in  the  tree  and  a   is  the  standard 

&  w 

deviation  of  the  weights  in  the  tree  and  a  is  an  adjustable 
parameter  and  is  set  equal  to  3  in  the  experiment.   Figure 
6.13  is  then  a  flow  diagram  which  not  only  gives  the  result 
of  the  appropriate  MST  but  also  gives  the  result  of  the 
clusters  of  the  MST.   Figure  6.14  is  the  MST  obtained  by 
this  method  and  the  corresponding  y  value  is  20.   Figure 
6.15  shows  the  clusters  of  the  MST. 

From  Figure  6.15  we  can  tell  that  the  nuclei  in  the 
same  cluster  are  of  the  same  type.   Nucleus  1  is  a  basal 
nucleus.   Nuclei  3  and  4  are  basal  nuclei.   Nuclei  9  and  3 
are  basal  nuclei.   Nucleus  2  is  a  squamous  nucleus.   Nuclei 
5,  6  and  7  are  squamous  nuclei. 

6.3.   Experiments  on  Blood  Cell  Pictures 

The  histogram  of  blood  cell  density  via  the  cell  in- 
tensity reveals  information  for  some  disease  diagnosis. 
Blood  cell  pictures  are  different  from  skin  cell  pictures 
in  that  there  are  some  blood  cells  whose  intensities  are 
lower  than  their  corresponding  background  intensities,  while 
every  skin  cell  intensity  is  higher  than  its  corresponding 
background  intensity.   Hence  a  hole  can  be  a  cell  in  the 
blood  cell  pictures.   Figure  6.16  is  a  portion  of  the  blood 
cell  pictures  in  our  experiment. 
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35.46158 


14.93248 


33.55661 


Figure  6.14.  An  MST  used  to  describe  the  distribution  of 
objects  in  Figure  6.10. 
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Figure  6.15.   The  clusters  of  objects  in  Figure  6.10 
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Figure  6.16.   An  8-level  picture  of  a  portion  of  a  blood 
cell  picture. 
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The  intensity  of  a  cell  is  defined  as  the  average  of 
the  intensities  of  picture  points  inside  the  cell.   Let  H 
be  the  Hasse  graph  of  all  the  contours  in  the  picture  which 
represent  the  inclusion  relation.   Let  C  be  the  object  con- 
tour of  a  cell  0  and  Hc  be  the  subtree  of  H  with  C  as  the 
root.   A  level  assignment  to  the  subtree  Hc  is  very  easily 
done  from  the  level  assignment  of  H.   Assume  that  the  level 
of  C  in  H  is  L(C),  the  level  of  any  contour  C  in  Hf  is  then 
L(C')  -  L(C)  +  1,  where  L(C')  is  the  level  of  C  in  H.   The 
intensity  of  the  cell  0  is  then 


n0  m,  m1 


o 
.2    *   CACC|)  -   EJ  ACcf1'^))  g(ch 

g(o)  =  i=Li=i — i w : l 

A(C) 

where  nQ  is  the  highest  level  of  the  subtree  Hc,  m.  is  the 
number  of  level  i  nodes  in  Hc ,  A(C')  is  the  area  enclosed 
by  the  contour  C,  C1.    is  the  contour  corresponding  to  a 
level  i  node  in  Hc ,  mj  is  the  number  of  sons  of  the  node 
corresponding  to  the  contour  c]   and  c/1'^  is  a  contour 
corresponding  to  a  son  of  the  node  corresponding  to  the 
contour  C^.  .   In  this  special  experiment,  the  intensity  of 
the  cell  is  assigned  as  the  integer  which  is  closest  to  the 
value  g(0)  calculated  from  the  above  equation.   The  count- 
ing of  overlapping  cells  is  done  by  the  flow  diagram  shown 
in  Figure  6.17.   K  in  the  flow  diagram  is  the  number  of 
cells  which  jointly  form  a  single  object  boundary  found  by 
the  contour  analysis. 
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Figure    6.17.       Flow   diagram   for   counting   overlapping 
blood   cells. 
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Figure  6.18  shows  the  histogram  of  intensities  of  a 
blood  cell  picture  in  our  experiment.  In  Figure  6.18,  n 
represents  the  total  number  of  cells. 

6.4.   Conclusions  and  Further  Research 

From  the  experiments,  it  is  found  that  the  two  methods 
of  object  extraction  are  good  in  different  situations: 

1.  Object  extraction  by  the  gradient  method  can  work  very- 
well  if  the  boundary  points  have  very  high  gradients.   That 
is,  it  works  well  if  the  boundaries  are  sharp. 

2.  Object  extraction  by  contour  analysis  works  well  under 
the  condition  that  the  objects  in  a  picture  occupy  quite 
uniform  intensity  regions.   This  method  is  especially  good 
for  area  pictures. 

By  using  the  graph-theory  approach  to  picture  proces- 
sing, a  clustering  method  can  then  be  applied.   A  multi- 
level picture  g  can  be  transferred  into  a  weighted  graph  G. 
From  the  weighted  graph  G,  a  minimal  spanning  forest  of  G 
can  then  be  found.   The  clustering  method  is  then  applied 
to  every  MST.   Let  rQ  be  the  radius  of  an  MST  T.   Small 
branches  in  T  are  removed  if  their  lengths  are  less  than 
arQ ,  where  o<a<l  is  a  threshold.   The  remaining  radial  paths 
are  combined  to  form  major  paths  by  the  method  stated  in 

Section  4.1.2.   Let  s.  =  (v.  ,...,v-  )  be  a  major  path  of 

o       t 
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Figure  6.18.   The  histogram  of  the  intensities  of  a  blood 
cell  picture. 
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length  t  and  (w.  ,...,w.  )  be  the  corresponding  weight  se- 

xl      xt 
quence,  that  is,  w.   is  the  weight  of  the  edge  connecting 

Xj 
v-       and  v-  .   Significant  local  minima  weights  are 

I(j-D        J 

noted  from  the  histograms  of  the  weights  along  the  major 

paths.   Several  cluster  ranges  of  weights  can  be  found  be- 
tween significant  local  minima.   If  w.   and  w.       are  in 

1 .       1 , .    ,  -, 
3  (3-1) 

different  cluster  ranges  and  if  w.   <  w.      ,  then  the  edge 

Xj     1(j-D 

connecting  vertices  v-      and  v.   are  broken.   After  this 

clustering  method  is  applied  to  all  major  paths  in  T,  T 

will  be  clustered  into  several  subtrees.   Hence  the  minimal 

spanning  forest  will  be  clustered  into  several  trees.   Let 

two  clusters  be  represented  by  two  trees  Tn  and  T„ .   T-,  and 

1      2  1 

T~  will  be  linked  together  if  (1)  the  average  weights  w, 
and  w~  of  T.  and  T-  are  very  close,  that  is,  |w, -w~|  <  w  , 
and  (2)  if  a  leaf  v,  in  T.,  and  a  leaf  v~  in  T~  are  adjacent 

in  GT,  and  if  the  weight  w'  of  the  edge  incident  with  v,  and 

wl+w2  |Wl+w2      i 

v^  are  very  close  to  — ^ —  >  "that  is  | — - —  -  w'  |  <  w  . 

After  this  linking  process,  every  tree  will  represent 
an  object  in  the  picture.   The  skeleton  of  an  object  can 
be  represented  by  the  significant  major  paths  of  the  tree 
representing  the  object. 

From  the  experiments  on  the  skin  cell  picture,  it  is 
found  that  object  extraction  by  contour  analysis  is  a  very 
promising  method  to  handle  cell  pictures.   The  global  fea- 
ture used  in  the  experiment  also  shows  that  the  description 
of  the  distribution  of  nuclei  in  the  picture  is  a  very  rea- 
sonable global  feature. 
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It  is  suggested  that  a  further  development  of  the 
system  is  needed  in  which  a  slide  is  taken  as  a  roll  of 
films  and  the  PIDAC  can  handle  a  roll  of  films.   It  is  pos- 
sible to  work  on  the  individual  film  and  to  store  the  local 
features  of  objects  in  the  individual  films.   In  describing 
the  global  features,  it  is  possible  to  consider  all  films 
at  one  time  by  knowing  the  location  of  each  film  in  the 
slide.   Once  such  a  system  is  set,  the  distribution  of  the 
nuclei  in  the  picture  can  be  found  more  completely. 

The  completion  of  the  clustering  stated  in  Section  6.2 
can  be  done  by  linking  together  the  nuclei  which  definitely 
belong  to  the  same  type.   When  one  scans  through  the  picture, 
if  the  object  contour  point  of  nucleus  0  is  always  just 
ahead  of  a  contour  point  of  a  high  level  contour  C  and  if 
another  nucleus  0'  satisfies  the  same  condition  for  the  same 
contour  C,  nuclei  0  and  0'  are  said  to  be  the  same  type. 
Let  each  cluster  obtained  by  the  method  stated  in  Section 
6.2  be  considered  as  a  vertex.   Clusters  S.  and  S.  are  con- 
nected if  there  exists  the  definite  same  type  nuclei  0,  , 
and  0^ , in  S.  and  S.,  respectively.   A  weight  W(S- ,S.)  is 
then  assigned  to  the  edge  connecting  S-  and  S.  by 

W(Si,Sj)  =  D(°k'°i)  =  min{D(oj,,ojt)|o£t  e  S±, 

o] ,  e  S.  and  0,1 ,  and  0^,  are  definitely 

£'     j      k'      I '  ' 

the  same  type}. 
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A  weighted  graph  is  then  set.   Find  the  corresponding  mini- 
mal spanning  forest.   If  the  clusters  S-  and  S.  are  con- 
nected in  the  minimal  spanning  forest,  connect  0,  and  Oi 
in  the  graph  obtained  through  the  cluster  shown  in  Section 
6.2. 

After  this  process,  it  is  easily  seen  that  for  a  normal 
epidermis,  there  should  be  two  large  clusters.  An  island 
of  nuclei  can  be  found  to  be  a  cluster  through  the  distri- 
bution of  the  objects  in  a  picture  by  considering  only  the 
distance  between  objects  (i.e.,  y  =  0) .  A  tumor  detection 
system  can  then  be  set  by  observing  the  resulting  clusters 
of  nuclei  in  an  island. 

The  blood  cell  extraction  is  more  complicated  than  the 
skin  cell  extraction  because  some  blood  cells  appear  as 
holes  in  the  picture.   While  the  result  we  get  in  the  blood 
cell  picture  analysis  is  quite  reasonable,  a  more  detailed 
research  should  be  conducted  to  achieve  a  more  complete  ob- 
ject extraction  system  in  dealing  with  the  pictures  which 
have  holes  as  objects.   Again  the  PIDAC  should  be  developed 
to  be  able  to  handle  a  rool  of  films  which  can  then  find 
more  reasonable  data  about  the  density  of  the  cells,  because 
the  result  from  one  film  is  too  restricted.   The  average  of 
the  density  of  a  roll  of  films  can  give  a  very  satisfactory 
result. 
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